-- Logs begin at Sun 2022-07-31 00:10:28 UTC, end at Sun 2022-07-31 01:33:02 UTC. -- Jul 31 00:58:56.218487 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 31 00:58:56.225934 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 31 00:58:56 2022] *** Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: nodename: dv-cinder-master-3-iscsi Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: machine: x86_64 Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: clock source: unix Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: pcre jit disabled Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: detected number of CPU cores: 4 Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: current working directory: / Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: detected binary path: /usr/bin/uwsgi-core Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: your processes number limit is 47680 Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: your memory page size is 4096 bytes Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: detected max file descriptor number: 2048 Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: lock engine: pthread robust mutexes Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: thunder lock: enabled Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 31 00:58:56.232979 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 31 00:58:56.269266 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: Python main interpreter initialized at 0x561abeedf9c0 Jul 31 00:58:56.269266 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: python threads support enabled Jul 31 00:58:56.269266 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: your server socket listen backlog is limited to 100 connections Jul 31 00:58:56.269266 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: your mercy for graceful operations on workers is 90 seconds Jul 31 00:58:56.269655 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: mapped 403077 bytes (393 KB) for 2 cores Jul 31 00:58:56.269731 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: *** Operational MODE: preforking *** Jul 31 00:58:56.269947 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: *** uWSGI is running in multiple interpreter mode *** Jul 31 00:58:56.269947 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: spawned uWSGI master process (pid: 103415) Jul 31 00:58:56.270157 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 31 00:58:56.270820 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: spawned uWSGI worker 1 (pid: 103416, cores: 1) Jul 31 00:58:56.271070 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: spawned uWSGI worker 2 (pid: 103417, cores: 1) Jul 31 00:58:56.271230 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 31 00:58:56.936247 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 31 00:58:56.937169 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 31 00:58:58.021893 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 00:58:58.021893 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: from cryptography.utils import int_from_bytes Jul 31 00:58:58.022619 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 00:58:58.022619 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: from cryptography.utils import int_from_bytes Jul 31 00:58:58.023797 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 00:58:58.023797 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: from cryptography.utils import int_from_bytes Jul 31 00:58:58.024221 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 00:58:58.024221 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: from cryptography.utils import int_from_bytes Jul 31 00:59:00.558982 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 00:59:00.559564 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 00:59:00.559806 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 00:59:00.560172 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 00:59:00.560430 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 00:59:00.560819 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.561146 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.561465 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 00:59:00.561827 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.561905 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 00:59:00.561977 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.562103 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.562286 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 00:59:00.562407 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.562594 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 00:59:00.562720 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.562848 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 00:59:00.563043 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.563233 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.563390 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.563584 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.563692 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.563901 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.563990 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.564205 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.564306 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.564596 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.565361 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.565546 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.565646 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.565727 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.565811 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.566028 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.566123 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.566350 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.566551 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.566634 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.566708 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.566907 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.567010 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.567226 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.567473 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.567877 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.567877 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.568124 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.568244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.568357 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.568477 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.568595 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.568734 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.568849 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.569052 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.569142 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.569352 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.569447 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.569828 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.569937 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.570171 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.570267 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.570378 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.570480 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.570690 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.570786 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.571035 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.571122 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.571346 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.571531 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.571668 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.571767 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.571980 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.572085 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.572282 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.572430 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.572600 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.572742 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.572904 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.573052 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.573303 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.573397 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.573611 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.573711 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.573919 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.574011 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.574233 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.574344 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.574600 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.574699 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.574918 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.575012 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.575227 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.575325 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.575553 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.575650 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.575860 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.575946 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.576170 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.576263 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.576495 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.576654 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.576796 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.576906 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.577100 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.577193 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.577356 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.577450 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.577646 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.577746 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.577967 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.578068 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.578275 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.578361 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.578589 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.578675 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.578840 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.578935 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.579090 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.579223 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.579436 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.579562 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.579778 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.579905 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.580091 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.580181 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.580420 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.580537 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.580692 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.580832 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.580940 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.581168 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.581266 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.581478 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.581609 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.581791 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.581885 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.582097 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.582195 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.582397 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.582492 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.582698 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.582796 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.583001 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.583088 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.583302 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.583425 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.583627 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.583721 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.583927 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.584044 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.584228 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.584444 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.584542 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.584696 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.584835 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.585004 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.585155 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.585346 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.585441 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.585664 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.585752 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.585974 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.586130 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.586293 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.586383 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.586616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.586721 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.586922 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.587015 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.587228 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.587315 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.587559 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.587659 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.587879 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.587966 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.588194 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.588287 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.588497 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.588596 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.588746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.588837 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.589048 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.589141 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.589355 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.589451 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.589678 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.589765 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.589983 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.590075 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.590229 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.590332 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.590528 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.590615 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.590829 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.590916 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.591140 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.591235 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.591470 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.591572 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.591774 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.591870 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.592075 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.592226 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.592368 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.592461 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.592668 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.592759 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.592968 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.593061 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.593267 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.593353 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.593566 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.593651 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.593866 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.593953 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.594162 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.594248 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.594461 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.594547 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.594761 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.594847 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.595059 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.595145 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.595390 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.595491 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.596303 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.596412 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.596525 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.596635 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.596820 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.596942 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.597116 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.597269 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.597416 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.597579 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.597727 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.597888 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.598026 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.598204 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.598325 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.598510 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.598637 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.598817 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.598936 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.599136 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.599223 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.599460 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.599552 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.599769 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.599867 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.600086 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.600172 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.600397 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.600493 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.600702 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.600798 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.600998 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.601083 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.601302 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.601386 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.601599 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.601686 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.601899 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.601987 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.602198 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.602330 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.602494 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.602592 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.602791 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.602884 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.603085 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.603171 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.603405 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.603517 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.603706 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.603802 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.603999 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.604108 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.604311 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.604425 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.604622 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.604719 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.604923 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.605044 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.605224 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.605323 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.605520 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.605618 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.605838 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 00:59:00.605939 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.606170 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.606269 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.606497 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.606589 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.606836 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.606941 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.607141 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.607233 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.607476 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.607569 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.607803 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.607903 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.608116 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.608211 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.608423 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.608523 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.608733 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.608825 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.609051 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.609150 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.609357 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.609447 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.609663 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.609755 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.609971 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.610084 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.610289 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.610390 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.610606 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.610706 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.610920 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.611010 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.611233 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.611941 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.612112 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.612204 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.612429 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.612541 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.612736 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.612824 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.613051 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.613140 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.613375 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.613469 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.613691 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.613799 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.613998 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.614085 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.614331 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.614423 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.614646 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.614741 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.614959 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.615045 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.615274 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.615399 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.615614 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.615716 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.615929 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.616016 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.616238 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.616331 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.616544 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.616638 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.616857 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.616965 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.617174 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.617275 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.617488 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.617583 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.617787 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.617884 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.618095 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.618196 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.618416 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.618518 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.618701 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.618797 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.619008 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.619110 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.619315 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.619442 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.619658 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.619754 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.619967 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.620061 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.620278 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.620373 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.620586 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.620680 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.620892 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.621000 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.621198 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.621299 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.621503 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.621596 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.621810 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.621915 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.622111 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.622206 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.622411 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.622521 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.622712 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.622820 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.623017 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.623119 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.623316 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.623431 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.623658 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.623766 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.623972 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.624066 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.624271 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.624365 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.624585 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.624678 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.624887 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.624981 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.625195 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.625296 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.625501 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.625597 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.625806 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.625901 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.626104 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.626197 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.626410 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.626506 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.626717 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.626815 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.627041 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.627161 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.627361 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.627523 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.627686 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.627780 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.627986 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.628098 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.628301 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.628387 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.628599 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.628683 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.628899 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.628994 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.629207 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.629294 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.629514 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.629602 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.629819 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.629914 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.630126 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.630213 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.630432 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.630518 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.630749 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.630852 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.631056 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.631157 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.631384 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.631473 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.631710 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.631797 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.632028 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.632154 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.632325 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.632408 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.632640 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.632736 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.632948 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.633033 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.633258 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.633346 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.633558 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.633659 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.633864 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.633950 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.634163 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.634254 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.634464 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.634549 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.634761 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.634845 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.635064 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.635160 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.635374 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.635463 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.635683 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.635785 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.635980 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.636074 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.636276 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.636362 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.636575 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.636661 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.636889 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.637072 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.637242 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.637502 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.637601 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.637703 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.637835 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.637937 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.638019 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.638174 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.638353 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.638499 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.638651 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.638824 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.638988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.639137 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.639316 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.639480 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.639671 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.639795 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.639991 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.640108 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.640299 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.640424 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.640612 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.640737 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.640919 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.641058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.641227 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.641371 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.641532 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.641691 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.641838 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.642004 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.642152 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.642326 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.642458 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.642718 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.642816 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.642986 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.643093 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.643295 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.643434 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.643639 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.643746 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.643982 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.644068 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.644315 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.644400 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.644633 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.644723 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.644957 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.645047 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.645270 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.645355 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.645581 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.645664 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.645894 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.645978 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.646198 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.646284 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.646507 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.646597 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.646810 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.646908 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.647117 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.647215 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.647452 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.647594 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.647758 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.647851 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.648070 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.648156 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.648375 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.648459 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.648688 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.648785 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.648994 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.649088 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.649301 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.649385 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.649604 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.649696 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.649918 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.650011 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.650216 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.650301 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.650525 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.650618 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.650845 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.650930 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.651166 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.651256 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.651492 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.651582 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.651801 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.651886 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.652117 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.652204 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.652421 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.652508 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.652724 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.652866 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.653032 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.653125 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.653332 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.653418 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.653631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.653715 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.653937 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.654022 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.654243 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.654335 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.654546 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.654637 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.654851 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.654954 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.655150 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.655235 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.655478 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.655564 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.655788 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.655887 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.656089 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.656174 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.656392 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.656484 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.656701 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.656798 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.657019 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.657121 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.657326 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.657436 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.657634 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.657740 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.657945 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.658030 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.658777 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.658872 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.659076 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.659168 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.659398 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.659486 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.659709 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.659794 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.660008 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.660093 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.660309 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.660403 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.660609 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.660695 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.660944 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.661038 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.661252 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.661337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.661553 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.661646 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.661866 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.661952 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.662176 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.662259 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.662479 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.662570 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.662789 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.662874 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.663096 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.663193 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.663423 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.663525 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.663735 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.663822 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.664043 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.664126 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.664347 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.664446 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.664653 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.664739 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.664960 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.665045 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.665266 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.665359 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.665570 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.665661 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.665879 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.665962 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.666185 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.666270 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.666490 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.666579 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.666795 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.666888 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.667095 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.667390 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.667500 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.667755 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.667852 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.668042 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.668282 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.668370 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.668610 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.668695 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.668922 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.669007 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.669247 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.669339 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.669801 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.669889 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.670115 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.670214 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.670438 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.670537 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.670761 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.670846 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.671075 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.671157 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.671404 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.671613 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.671711 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.671790 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672002 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672089 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672302 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672395 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672608 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672696 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672906 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.672991 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.673208 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.673305 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.673509 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.673601 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.673858 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.674474 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.674662 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.674760 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.674965 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.675051 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.675265 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.675381 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.675601 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.675699 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.675908 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.676000 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.676214 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.676299 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.676524 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.676622 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.676822 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.676909 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.677126 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.677217 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.677434 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.677538 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.677734 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.677842 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.678033 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.678143 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.678332 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.678419 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.678671 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.678757 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.678972 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.679103 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.679303 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.679423 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.679647 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.679743 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.679976 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.680062 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.680275 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.680397 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.680577 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.680664 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.680886 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.680979 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.681185 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.681268 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.681485 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.681582 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.681784 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.681885 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.682087 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.682174 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.682390 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.682476 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.682715 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.682820 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.683084 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.683216 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.683302 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.683429 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.683635 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.683736 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.683924 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.684013 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.684195 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.684316 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.684499 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.684585 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.684798 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.684884 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.685103 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.685186 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.685406 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.685497 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.685704 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.685790 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.686003 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.686089 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.686312 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.686399 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.686613 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.686696 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.686910 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.687002 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.687216 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.687302 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.687543 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.687857 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.687950 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.688162 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.688253 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.688459 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.688560 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.688754 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.688883 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.689057 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.689194 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.689362 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.689549 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.689686 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.689866 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.690003 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.690174 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.690308 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.690482 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.690607 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.690801 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.690895 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.691108 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.691193 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.691442 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.691549 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.691759 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.691857 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.692080 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.692165 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.692401 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.692541 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.692706 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.692797 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693002 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693086 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693307 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693392 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693606 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693700 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693905 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.693990 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.694203 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.694295 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.694504 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.694589 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.694799 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.694883 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.695097 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.695191 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.695416 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.695507 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.695719 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.695818 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.696016 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.696115 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.696337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.696421 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.696638 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.696729 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.696937 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.697031 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.697237 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.697324 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.697533 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.697675 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.697868 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.698010 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.698171 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.698356 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.698520 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.698665 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.698821 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.698982 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.699130 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.699305 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.699466 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.699645 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.699816 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.699960 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.700055 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.700274 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.700360 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.700585 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.700680 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.700901 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.700986 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.701213 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.701300 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.701527 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.701621 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.701824 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.701914 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.702124 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.702209 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.702439 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.702537 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.702746 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.702832 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.703051 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.703144 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.703398 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.703491 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.703700 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.703786 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.704014 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.704097 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.704314 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.704399 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.704616 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.704701 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.704921 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.705009 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.705249 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.705335 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.705551 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.705637 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.705855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.705941 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.706164 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.706248 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.706469 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.706553 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.706774 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.706868 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.707081 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.707165 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.707411 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.707523 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.707719 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.707855 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.708028 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.708149 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.708337 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.708431 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.708639 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.708747 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.708955 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.709039 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.709261 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.709346 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.709562 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.709661 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.709868 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.709952 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.710166 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.710251 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.710471 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.710557 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.710770 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.710856 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.711192 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.711285 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.711525 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.711613 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.711833 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.711919 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.712135 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.712237 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.712441 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.712528 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.712747 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.712858 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.713059 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.713145 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.713367 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.713453 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.713667 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.713755 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.714073 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.714172 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.714381 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.714467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.714685 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.714770 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.714988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.715083 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.715289 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.715393 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.715611 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.715706 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.715916 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.716015 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.716237 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.716333 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.716535 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.716627 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.716836 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.716932 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.717131 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.717215 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.717448 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.717533 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.717750 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.717834 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.718064 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.718151 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.718473 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.718571 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.718780 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.718867 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.719080 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.719166 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.719520 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.719625 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.719828 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.719961 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.720137 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.720265 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.720452 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.720611 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.721308 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.721442 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.721605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.721734 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.721908 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.722163 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.722306 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.722439 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.722562 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.722677 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.722813 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.722907 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.723124 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.723231 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.723462 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.723582 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.723770 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.723907 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.724067 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.724215 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.724382 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.724542 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.724698 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.724819 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.725007 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.725130 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.725325 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.725436 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.725734 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.725868 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.726051 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.726181 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.726357 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.726481 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.726680 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.726798 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.726999 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.727110 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.727301 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.727485 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.727626 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.727715 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.727919 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.728008 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.728219 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.728308 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.728522 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.728620 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.728835 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.728923 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.729133 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.729220 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.729444 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.729559 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.729750 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.729850 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.730055 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.730150 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.730354 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.730457 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.730654 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.730754 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.730969 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.731062 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.731269 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.731404 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.731610 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.731704 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.731919 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.732011 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.732290 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.732401 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.732548 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.732644 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.732847 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.732956 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.733168 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.733279 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.733473 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.733567 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.733771 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.733858 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.734080 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.734192 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.734370 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.734463 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.734706 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.734800 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.735007 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.735101 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.735324 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.735451 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.735669 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.735755 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.735991 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.736081 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.736316 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.736402 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.737170 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.737275 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.737491 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.737578 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.737800 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.737885 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.738112 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.738254 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.738432 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.738534 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.738731 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.738829 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.739025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.739120 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.739329 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.739482 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.739651 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.739765 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.739950 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.740153 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.740263 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.740376 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.740530 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.740843 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.741153 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.741456 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.741694 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.741781 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.741998 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.742083 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.742300 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.742383 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.742601 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.742687 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.742921 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.743061 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.743221 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.743320 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.743563 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.743663 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.743869 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.743955 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.744169 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.744263 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.744468 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.744562 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.744765 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.744864 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.745102 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.745189 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.745402 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.745489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.745730 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.745824 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.746044 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.746129 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.746362 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.746445 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.746679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.746777 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.746982 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.747074 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.747298 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.747404 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.747639 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.747743 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.747953 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.748037 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.748320 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.748429 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.748572 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.748668 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.748879 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.748976 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.749186 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.749271 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.749583 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.749686 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.749800 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.750071 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.750152 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.750402 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.750502 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.750713 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.750798 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.751032 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.751263 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.751385 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.751616 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.751713 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.751950 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.752043 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.752263 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.752358 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.752572 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.752657 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.752879 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.752964 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.753185 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.753285 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.753493 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.753579 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.753855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.753956 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.754106 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.754301 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.754412 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.754499 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.754737 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.754828 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.755052 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.755139 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.755379 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.755479 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.755708 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.755815 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.756028 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.756124 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.756333 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.756420 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.756642 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.756738 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.756951 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.757041 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.757266 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.757351 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.757583 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.757678 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.757885 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.757977 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.758192 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.758278 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.758502 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.758596 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.758811 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.758912 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.759120 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.759218 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.759446 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.759537 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.759756 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.759842 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.760064 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.760154 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.760373 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.760468 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.760678 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.760764 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.760986 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.761080 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.761293 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.761380 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.761614 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.761700 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.761931 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.762025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.762248 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.762458 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.762544 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.762780 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.762866 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.763134 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.763229 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.763475 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.763680 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.763905 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.764012 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.764156 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.764304 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.764481 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.764625 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.764814 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.764945 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.765125 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.765260 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.765433 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.765574 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.765748 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.765896 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.766060 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.766198 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.766366 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.766498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.766692 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.766806 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.767007 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.767112 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.767316 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.767433 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.767692 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.767797 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.768005 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.768091 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.768349 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.768568 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.768680 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.768891 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.768978 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.769212 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.769468 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.769556 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.769811 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.769907 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.770123 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.770209 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.770442 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.770537 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.770759 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.770853 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.771071 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.771164 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.771397 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.771487 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.771719 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.771808 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.772019 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.772103 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.772320 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.772414 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.772623 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.772708 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.772929 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.773017 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.773232 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.773331 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.773533 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.773619 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.773840 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.773926 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.774143 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.774241 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.774443 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.774527 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.774751 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.774838 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.775052 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.775146 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.775373 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.775460 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.775688 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.775774 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.775986 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.776073 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.776286 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.776371 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.776589 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.776673 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.776886 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.776973 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.777184 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.777269 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.777483 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.777569 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.777786 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.777873 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.778088 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.778173 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.778385 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.778469 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.778692 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.778790 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.778989 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.779075 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.779296 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.779447 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.779617 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.779706 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.779924 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.780011 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.780250 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.780344 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.780565 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.780664 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.780867 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.780951 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.781166 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.781261 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.781474 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.781566 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.781780 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.781866 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.782096 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.782195 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.782405 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.782498 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.782709 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.782950 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.783045 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.783835 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.783947 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.784160 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.784418 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.784548 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.784751 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.784845 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.785071 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.785156 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.785403 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.785493 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.785708 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.785793 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.786022 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.786105 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.786324 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.786409 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.786627 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.786720 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.786936 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.787021 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.787245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.787338 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.787579 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.787663 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.787891 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.787986 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.788201 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.788293 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.788510 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.788596 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.788820 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.788919 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.789125 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.789222 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.789436 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.789534 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.789741 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.789828 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.790047 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.790142 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.790355 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.790440 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.790672 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.790896 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.791002 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.791322 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.791502 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.791668 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.791987 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.792070 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.792307 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.792448 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.792620 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.792799 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.792930 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.793125 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.793239 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.793445 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.793548 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.793845 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.793984 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.794186 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.794671 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.794777 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.795051 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.795145 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.795397 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.795501 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.795714 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.795802 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.796025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.796119 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.796342 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.796439 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.796671 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.796759 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.796986 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.797075 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.797347 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.797516 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.797659 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.797986 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.798284 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.798371 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.798872 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.799617 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.799710 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.799972 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.800059 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.800300 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.800588 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.800731 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.800909 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.801105 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.801275 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.801521 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.801637 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.801751 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.801838 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.802069 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.802146 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.802395 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.802480 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.802702 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.802791 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.803014 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.803089 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.803319 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.803432 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.803647 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.803729 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.803958 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.804215 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.804303 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.804602 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.804780 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.804869 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.805169 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.805327 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.805446 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.805748 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.805888 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.806052 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.806355 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.806458 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.806664 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.806790 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.806970 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.807275 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.807497 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.807600 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.807690 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.807892 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.807979 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.808194 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.808288 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.808526 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.808613 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.808837 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.808922 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.809139 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.809234 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.809441 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.809535 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.809754 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.809852 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.810053 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.810151 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.810354 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.810441 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.810655 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.810741 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.810955 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.811054 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.811253 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.811346 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.811578 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.811665 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.811885 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.811973 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.812187 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.812272 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.812487 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.812574 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.812787 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.812874 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.813086 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.813171 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.813382 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.813467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.813709 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.813797 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.814038 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.814135 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.814350 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.814466 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.814696 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.814791 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.815003 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.815089 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.815321 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.815439 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.815652 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.815738 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.815970 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.816057 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.816282 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.816375 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.816592 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.816676 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.816899 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.816984 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.817214 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.817310 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.817525 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.817610 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.817835 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.817923 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.818143 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.818272 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.818452 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.818549 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.818763 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.818848 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.819069 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.819155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.819408 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.819497 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.819744 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.819874 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.820032 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.820118 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.820350 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.820436 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.820666 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.820761 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.820983 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.821069 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.821296 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.821389 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.821602 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.821697 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.821928 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.822014 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.822234 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.822322 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.822543 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.822637 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.822851 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.822937 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.823160 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.823253 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.823498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.823587 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.823731 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103416) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 00:59:00.823854 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.824149 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.824418 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.824726 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.825041 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.825363 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.825685 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.825992 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.826314 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.826636 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.826976 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.827286 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.827655 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.827980 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.828292 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.828602 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.828907 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 00:59:00.829147 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 00:59:00.978401 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO dbcounter [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Registered counter for database nova_api Jul 31 00:59:00.982363 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO dbcounter [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Registered counter for database nova_api Jul 31 00:59:01.000175 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_db.sqlalchemy.engines [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.003092 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_db.sqlalchemy.engines [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.007725 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writer thread running {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.009559 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writer thread running {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.064998 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.context [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d(cell1) {{(pid=103416) load_cells /opt/stack/nova/nova/context.py:464}} Jul 31 00:59:01.065195 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.context [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d(cell1) {{(pid=103417) load_cells /opt/stack/nova/nova/context.py:464}} Jul 31 00:59:01.069400 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 00:59:01.069512 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_concurrency.lockutils [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103416) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 00:59:01.070962 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 00:59:01.071079 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_concurrency.lockutils [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103416) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 00:59:01.071787 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 00:59:01.071883 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_concurrency.lockutils [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103416) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 00:59:01.073753 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 00:59:01.073877 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_concurrency.lockutils [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103416) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 00:59:01.075731 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO dbcounter [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Registered counter for database nova_cell0 Jul 31 00:59:01.075846 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO dbcounter [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Registered counter for database nova_cell0 Jul 31 00:59:01.081237 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO dbcounter [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Registered counter for database nova_cell1 Jul 31 00:59:01.081371 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO dbcounter [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Registered counter for database nova_cell1 Jul 31 00:59:01.088883 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_db.sqlalchemy.engines [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.091756 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_db.sqlalchemy.engines [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.091897 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_db.sqlalchemy.engines [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.095167 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_db.sqlalchemy.engines [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.096483 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writer thread running {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.098607 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writer thread running {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.099192 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writer thread running {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.240685 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writer thread running {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.248964 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO dbcounter [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Registered counter for database nova_cell0 Jul 31 00:59:01.251708 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO dbcounter [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Registered counter for database nova_cell0 Jul 31 00:59:01.259660 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_db.sqlalchemy.engines [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.260803 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG oslo_db.sqlalchemy.engines [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103416) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 00:59:01.264576 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writer thread running {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.265610 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writer thread running {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 00:59:01.607847 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: WARNING keystonemiddleware.auth_token [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 00:59:01.608453 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: WARNING keystonemiddleware.auth_token [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 00:59:01.616163 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: WARNING keystonemiddleware.auth_token [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 00:59:01.616360 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: WARNING keystonemiddleware.auth_token [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 00:59:01.958543 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: WARNING keystonemiddleware.auth_token [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 00:59:01.960171 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: WARNING keystonemiddleware.auth_token [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 00:59:01.964539 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: WARNING keystonemiddleware.auth_token [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 00:59:01.966087 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: WARNING keystonemiddleware.auth_token [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 00:59:01.969799 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561abeedf9c0 pid: 103416 (default app) Jul 31 00:59:01.971977 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561abeedf9c0 pid: 103417 (default app) Jul 31 00:59:01.972492 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 00:59:01.973673 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-809eb51e-548e-47cc-9b88-d097c6c04bbb None None] 10.222.0.42 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001749 Jul 31 00:59:01.974088 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 788 bytes} [Sun Jul 31 00:59:01 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:18.475560 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:18.477204 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002395 Jul 31 01:00:18.477831 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:18 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:18.494401 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writing DB stats nova_api:SELECT=3 {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:18.495311 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writing DB stats nova_cell0:SELECT=3 {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:18.496831 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writing DB stats nova_cell1:SELECT=3 {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:18.498146 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:19.428152 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-9ac246f6-fef4-4eb6-87a9-1bd4e1e98b6f admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:19.460472 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-9ac246f6-fef4-4eb6-87a9-1bd4e1e98b6f admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.971988 Jul 31 01:00:19.461229 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 2/3] 10.222.0.42 () {60 vars in 1291 bytes} [Sun Jul 31 01:00:18 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 973 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:21.811717 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:21.812799 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-8cc3101f-564d-48a3-aa1d-b12a25a9a355 None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002067 Jul 31 01:00:21.818313 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 2/4] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:21 2022] GET /compute/v2.1 => generated 388 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:22.290571 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-1b616f0b-f562-44fc-ad42-6e16411584cc admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "swap": 0, "id": "c1", "name": "cirros256", "rxtx_factor": 1.0, "vcpus": 1, "disk": 1, "os-flavor-access:is_public": true}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:22.312177 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:00:22.312177 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: warnings.warn(msg) Jul 31 01:00:22.343726 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-1b616f0b-f562-44fc-ad42-6e16411584cc admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.523589 Jul 31 01:00:22.344656 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 3/5] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:21 2022] POST /compute/v2.1/flavors => generated 407 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:00:22.359795 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_api:SELECT=3 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:22.360922 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_cell0:SELECT=3 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:22.361827 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_cell1:SELECT=3 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:22.363102 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:22.583627 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-c6376dd0-1ecd-4e5c-b731-5b14e77b7981 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:22.587981 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:00:22.587981 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: warnings.warn(msg) Jul 31 01:00:22.660642 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-c6376dd0-1ecd-4e5c-b731-5b14e77b7981 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.307366 Jul 31 01:00:22.668949 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 3/6] 10.222.0.42 () {64 vars in 1335 bytes} [Sun Jul 31 01:00:22 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:24.932641 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-1b616f0b-f562-44fc-ad42-6e16411584cc admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:24.933663 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-1b616f0b-f562-44fc-ad42-6e16411584cc admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001470 Jul 31 01:00:24.934325 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 4/7] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:24 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:00:25.698302 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-d50f35e5-6f85-4f74-9b1e-954434851427 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 5, "name": "ds512M", "os-flavor-access:is_public": true, "swap": 0, "id": "d1", "ram": 512, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:25.719679 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:00:25.719679 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: warnings.warn(msg) Jul 31 01:00:25.740220 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-d50f35e5-6f85-4f74-9b1e-954434851427 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.797106 Jul 31 01:00:25.747929 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 4/8] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:24 2022] POST /compute/v2.1/flavors => generated 404 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:00:25.753196 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-b5d29696-eb08-464f-8ce4-d02cdedd7eea admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:25.756227 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:00:25.756227 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: warnings.warn(msg) Jul 31 01:00:25.792675 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-b5d29696-eb08-464f-8ce4-d02cdedd7eea admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.043208 Jul 31 01:00:25.793361 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 5/9] 10.222.0.42 () {64 vars in 1335 bytes} [Sun Jul 31 01:00:25 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:28.172252 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-d50f35e5-6f85-4f74-9b1e-954434851427 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:28.173350 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-d50f35e5-6f85-4f74-9b1e-954434851427 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002096 Jul 31 01:00:28.173824 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:28 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:00:28.629655 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-b3d63475-3816-4640-bc5d-d5021142f887 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 10, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "id": "d2", "name": "ds1G", "os-flavor-access:is_public": true, "ram": 1024}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:28.649712 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-b3d63475-3816-4640-bc5d-d5021142f887 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.469608 Jul 31 01:00:28.650191 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 6/11] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:28 2022] POST /compute/v2.1/flavors => generated 404 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:00:28.662894 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-f23c61fb-80ad-435c-92a9-76791020907f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:28.694291 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-f23c61fb-80ad-435c-92a9-76791020907f admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035144 Jul 31 01:00:28.694980 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 6/12] 10.222.0.42 () {64 vars in 1335 bytes} [Sun Jul 31 01:00:28 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:30.949032 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-b3d63475-3816-4640-bc5d-d5021142f887 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:30.950063 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-b3d63475-3816-4640-bc5d-d5021142f887 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001481 Jul 31 01:00:30.950480 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 7/13] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:31.447130 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-5c5643dd-7e10-4c1b-9540-00b3a116d05b admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "vcpus": 2, "ram": 2048, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d3", "swap": 0, "name": "ds2G", "rxtx_factor": 1.0}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:31.477376 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-5c5643dd-7e10-4c1b-9540-00b3a116d05b admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.519546 Jul 31 01:00:31.478839 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 7/14] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:30 2022] POST /compute/v2.1/flavors => generated 404 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:00:31.490684 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-92e12e21-da24-452a-96db-041882262d12 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:31.520633 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-92e12e21-da24-452a-96db-041882262d12 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033522 Jul 31 01:00:31.521642 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 8/15] 10.222.0.42 () {64 vars in 1335 bytes} [Sun Jul 31 01:00:31 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:00:33.775849 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-5c5643dd-7e10-4c1b-9540-00b3a116d05b admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:33.778589 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-5c5643dd-7e10-4c1b-9540-00b3a116d05b admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003187 Jul 31 01:00:33.778997 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 8/16] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:33 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:00:34.450979 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-e2eba6c5-ee46-4852-8e0c-d9cd99aaf492 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds4G", "id": "d4", "disk": 20, "ram": 4096, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "os-flavor-access:is_public": true, "vcpus": 4}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:34.473754 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-e2eba6c5-ee46-4852-8e0c-d9cd99aaf492 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.687551 Jul 31 01:00:34.474590 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 9/17] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:33 2022] POST /compute/v2.1/flavors => generated 404 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:00:34.487723 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-ebf11458-0aa2-4351-b1a2-579774c16130 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:34.512040 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-ebf11458-0aa2-4351-b1a2-579774c16130 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028466 Jul 31 01:00:34.512850 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 9/18] 10.222.0.42 () {64 vars in 1335 bytes} [Sun Jul 31 01:00:34 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:00:37.019961 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-e2eba6c5-ee46-4852-8e0c-d9cd99aaf492 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:37.021029 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-e2eba6c5-ee46-4852-8e0c-d9cd99aaf492 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001508 Jul 31 01:00:37.021436 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 10/19] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:37.447845 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-3afd440c-665f-47dd-9a83-db2db7e45ad0 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:37.472630 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-3afd440c-665f-47dd-9a83-db2db7e45ad0 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.445251 Jul 31 01:00:37.473162 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 10/20] 10.222.0.42 () {60 vars in 1291 bytes} [Sun Jul 31 01:00:37 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:00:39.753587 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-e2eba6c5-ee46-4852-8e0c-d9cd99aaf492 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:39.754531 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-e2eba6c5-ee46-4852-8e0c-d9cd99aaf492 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Jul 31 01:00:39.755004 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 11/21] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:39 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:40.177080 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-3f11b3d1-8898-4b0c-907b-23d015622d8b admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "id": "1", "vcpus": 1, "rxtx_factor": 1.0, "ram": 512, "disk": 1, "os-flavor-access:is_public": true, "name": "m1.tiny", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:40.196009 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-3f11b3d1-8898-4b0c-907b-23d015622d8b admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.434532 Jul 31 01:00:40.196569 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 11/22] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:39 2022] POST /compute/v2.1/flavors => generated 402 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:00:40.209098 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-9312f209-15aa-456e-8448-e36bab63e53b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:40.237205 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-9312f209-15aa-456e-8448-e36bab63e53b admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031441 Jul 31 01:00:40.238273 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 12/23] 10.222.0.42 () {64 vars in 1332 bytes} [Sun Jul 31 01:00:40 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:42.457388 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-3f11b3d1-8898-4b0c-907b-23d015622d8b admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:42.458402 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-3f11b3d1-8898-4b0c-907b-23d015622d8b admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Jul 31 01:00:42.458798 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:42.862252 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-d74550ed-17cf-46db-9036-73f575058cfa admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "2", "swap": 0, "os-flavor-access:is_public": true, "vcpus": 1, "name": "m1.small", "rxtx_factor": 1.0}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:42.882303 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-d74550ed-17cf-46db-9036-73f575058cfa admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.416699 Jul 31 01:00:42.882861 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 13/25] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:42 2022] POST /compute/v2.1/flavors => generated 405 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:00:42.894956 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-624b71ff-95bf-4262-a823-0ec1b2736fc7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:42.920024 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-624b71ff-95bf-4262-a823-0ec1b2736fc7 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028723 Jul 31 01:00:42.920856 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 13/26] 10.222.0.42 () {64 vars in 1332 bytes} [Sun Jul 31 01:00:42 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:45.135500 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-d74550ed-17cf-46db-9036-73f575058cfa admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:45.137896 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-d74550ed-17cf-46db-9036-73f575058cfa admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002542 Jul 31 01:00:45.138397 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 14/27] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:45 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:45.513664 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-237629c6-c95c-4e95-888d-95307e6494a9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "disk": 40, "ram": 4096, "vcpus": 2, "swap": 0, "os-flavor-access:is_public": true, "name": "m1.medium", "rxtx_factor": 1.0, "id": "3"}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:45.538748 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-237629c6-c95c-4e95-888d-95307e6494a9 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.393561 Jul 31 01:00:45.539956 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 14/28] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:45 2022] POST /compute/v2.1/flavors => generated 406 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:00:45.553751 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88ab7459-b1ac-4259-8c7a-4a958c6ec4ed admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:45.578834 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88ab7459-b1ac-4259-8c7a-4a958c6ec4ed admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029457 Jul 31 01:00:45.579767 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 15/29] 10.222.0.42 () {64 vars in 1332 bytes} [Sun Jul 31 01:00:45 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:47.818230 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-237629c6-c95c-4e95-888d-95307e6494a9 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:47.819955 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-237629c6-c95c-4e95-888d-95307e6494a9 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002166 Jul 31 01:00:47.820579 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 15/30] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:47 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:48.290941 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-fe5bca2e-15a9-4cdb-88c2-50c7098db0ab admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 80, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "id": "4", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "swap": 0, "ram": 8192, "vcpus": 4}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:48.313030 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-fe5bca2e-15a9-4cdb-88c2-50c7098db0ab admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.486592 Jul 31 01:00:48.313796 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 16/31] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:47 2022] POST /compute/v2.1/flavors => generated 405 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:00:48.325731 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-95852dfb-2280-448e-8df8-a2e726e3fd97 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:48.354302 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-95852dfb-2280-448e-8df8-a2e726e3fd97 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032439 Jul 31 01:00:48.355088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 16/32] 10.222.0.42 () {64 vars in 1332 bytes} [Sun Jul 31 01:00:48 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:00:50.607579 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-fe5bca2e-15a9-4cdb-88c2-50c7098db0ab admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:00:50.608590 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-fe5bca2e-15a9-4cdb-88c2-50c7098db0ab admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001491 Jul 31 01:00:50.608992 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 17/33] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:00:50 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:00:51.085571 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-fe63f818-756f-4624-b3ab-c119996ed3c9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "swap": 0, "vcpus": 8, "os-flavor-access:is_public": true, "name": "m1.xlarge", "id": "5", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 16384, "disk": 160}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:51.105802 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-fe63f818-756f-4624-b3ab-c119996ed3c9 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.489931 Jul 31 01:00:51.106360 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 17/34] 10.222.0.42 () {64 vars in 1282 bytes} [Sun Jul 31 01:00:50 2022] POST /compute/v2.1/flavors => generated 408 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:00:51.118965 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-0a36f4e4-8593-4cba-8dbd-84cf565fa411 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:00:51.125660 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=9,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:00:51.150998 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-0a36f4e4-8593-4cba-8dbd-84cf565fa411 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035720 Jul 31 01:00:51.151904 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 18/35] 10.222.0.42 () {64 vars in 1332 bytes} [Sun Jul 31 01:00:51 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:01:10.754061 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-fe63f818-756f-4624-b3ab-c119996ed3c9 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:10.755090 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-fe63f818-756f-4624-b3ab-c119996ed3c9 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Jul 31 01:01:10.755525 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 18/36] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:01:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:11.141276 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-6549ac64-a0ee-4717-b6f5-0068aabde684 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:11.160910 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-6549ac64-a0ee-4717-b6f5-0068aabde684 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.399573 Jul 31 01:01:11.161772 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 19/37] 10.222.0.42 () {60 vars in 1291 bytes} [Sun Jul 31 01:01:10 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:01:12.230602 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-fe63f818-756f-4624-b3ab-c119996ed3c9 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:12.231632 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-fe63f818-756f-4624-b3ab-c119996ed3c9 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001426 Jul 31 01:01:12.232028 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 19/38] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:12.753204 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-5068fe41-5332-48c4-ab16-239f02606cb8 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:12.758767 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.wsgi [None req-5068fe41-5332-48c4-ab16-239f02606cb8 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-5068fe41-5332-48c4-ab16-239f02606cb8 admin admin] Request method failure captured: Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Accept: application/json Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Accept-Encoding: gzip, deflate Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Connection: keep-alive Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Content-Length: 0 Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Host: 10.222.0.42 Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Openstack-Api-Version: compute 2.90 Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Openstack-System-Scope: None Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: User-Agent: python-novaclient Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Auth-Token: gAAAAABi5dRY7UFikWgPr_gZhmlU396DvcUa-r1RSxBnw_L6ky13sQsIvGRuGHGdzm4Eqnmt6WVHX4imuTb7h90cRyK1mcrk1Lh00JAc80i2NI7Nzdlz1hSl3tN1fA95s0gVfxIvFVnMhOB_nSW3Zz59xxrRQRHA25C-9IuAPBPTOq5I0WIBqRk Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Domain-Id: None Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Domain-Name: None Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Identity-Status: Confirmed Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Is-Admin-Project: True Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Openstack-Nova-Api-Version: 2.90 Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Domain-Id: default Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Domain-Name: Default Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Name: admin Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Role: reader,admin,member Jul 31 01:01:12.762058 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Roles: reader,admin,member Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_18e627c596bb4c99b66fda211d0e4785", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant: admin Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant-Name: admin Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User: admin Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Domain-Id: default Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Domain-Name: Default Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Id: e711c7fc76434a0b86b92dc8bcf69e87 Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Name: admin Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: method: > Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: exception: Flavor 42 could not be found. Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: action_args: {'id': '42'} Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:01:12.763679 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:12.765746 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.767155 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-5068fe41-5332-48c4-ab16-239f02606cb8 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103416) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:01:12.768774 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-5068fe41-5332-48c4-ab16-239f02606cb8 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.393820 Jul 31 01:01:12.768774 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 20/39] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Jul 31 01:01:12 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 395 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 31 01:01:12.773714 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-c15b2d3a-a838-4e13-aa74-8a0b02cf810a admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:12.777933 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:01:12.797348 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-c15b2d3a-a838-4e13-aa74-8a0b02cf810a admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027206 Jul 31 01:01:12.797902 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 20/40] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Jul 31 01:01:12 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:01:12.807897 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-ef3a32ac-4212-469e-8398-d1057b99eb07 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:12.832070 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-ef3a32ac-4212-469e-8398-d1057b99eb07 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.027692 Jul 31 01:01:12.832610 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 21/41] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Jul 31 01:01:12 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:01:12.841923 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-6d5707e9-e3b1-4473-831a-b7695ce8896c admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:12.848147 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.wsgi [None req-6d5707e9-e3b1-4473-831a-b7695ce8896c admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-6d5707e9-e3b1-4473-831a-b7695ce8896c admin admin] Request method failure captured: Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Accept: application/json Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Accept-Encoding: gzip, deflate Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Connection: keep-alive Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Content-Length: 0 Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Host: 10.222.0.42 Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Openstack-Api-Version: compute 2.90 Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Openstack-System-Scope: None Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: User-Agent: python-novaclient Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Auth-Token: gAAAAABi5dRY7UFikWgPr_gZhmlU396DvcUa-r1RSxBnw_L6ky13sQsIvGRuGHGdzm4Eqnmt6WVHX4imuTb7h90cRyK1mcrk1Lh00JAc80i2NI7Nzdlz1hSl3tN1fA95s0gVfxIvFVnMhOB_nSW3Zz59xxrRQRHA25C-9IuAPBPTOq5I0WIBqRk Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Domain-Id: None Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Domain-Name: None Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Identity-Status: Confirmed Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Is-Admin-Project: True Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Openstack-Nova-Api-Version: 2.90 Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Domain-Id: default Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Domain-Name: Default Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Name: admin Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Role: reader,admin,member Jul 31 01:01:12.851605 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Roles: reader,admin,member Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_18e627c596bb4c99b66fda211d0e4785", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant: admin Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant-Name: admin Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User: admin Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Domain-Id: default Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Domain-Name: Default Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Id: e711c7fc76434a0b86b92dc8bcf69e87 Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Name: admin Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: method: > Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: exception: Flavor 42 could not be found. Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: action_args: {'id': '42'} Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:01:12.853088 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:12.853988 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:12.855245 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-6d5707e9-e3b1-4473-831a-b7695ce8896c admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103417) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:01:12.856450 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-6d5707e9-e3b1-4473-831a-b7695ce8896c admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014715 Jul 31 01:01:12.856450 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 21/42] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Jul 31 01:01:12 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 31 01:01:13.823565 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-ef3a32ac-4212-469e-8398-d1057b99eb07 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:13.824574 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-ef3a32ac-4212-469e-8398-d1057b99eb07 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001481 Jul 31 01:01:13.824970 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 22/43] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:14.350450 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-1832c2ce-a70b-4fbe-b7d2-3f6dd4bb3547 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:14.355839 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.wsgi [None req-1832c2ce-a70b-4fbe-b7d2-3f6dd4bb3547 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-1832c2ce-a70b-4fbe-b7d2-3f6dd4bb3547 admin admin] Request method failure captured: Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Accept: application/json Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Accept-Encoding: gzip, deflate Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Connection: keep-alive Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Content-Length: 0 Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Host: 10.222.0.42 Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Openstack-Api-Version: compute 2.90 Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Openstack-System-Scope: None Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: User-Agent: python-novaclient Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Auth-Token: gAAAAABi5dRZmUbGd3OyyZpITtvY9uf7SwFOO-iOogRvdweSecf1pr3-r44qKlB8ylPhkRE8-bNp0KFrqjmq1GnDZl-xnG8NlqzDsG2Su8aSTc-xamBYUqN-wYp1FuDKKzP5LwXKVfjiwifMSgyL64ggj8haaxZpvZDzMJxgDUTmqFIKuAQkGxo Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Domain-Id: None Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Domain-Name: None Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Identity-Status: Confirmed Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Is-Admin-Project: True Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Openstack-Nova-Api-Version: 2.90 Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Domain-Id: default Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Domain-Name: Default Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Name: admin Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Role: reader,admin,member Jul 31 01:01:14.357330 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Roles: reader,admin,member Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_18e627c596bb4c99b66fda211d0e4785", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant: admin Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant-Name: admin Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User: admin Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Domain-Id: default Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Domain-Name: Default Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Id: e711c7fc76434a0b86b92dc8bcf69e87 Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Name: admin Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: method: > Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: exception: Flavor 84 could not be found. Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: action_args: {'id': '84'} Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:01:14.358855 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:14.359841 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.361025 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-1832c2ce-a70b-4fbe-b7d2-3f6dd4bb3547 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103417) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:01:14.362123 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-1832c2ce-a70b-4fbe-b7d2-3f6dd4bb3547 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.398331 Jul 31 01:01:14.362123 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 22/44] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Jul 31 01:01:13 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 399 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 31 01:01:14.368973 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-a449d127-1fbb-4c2b-9a12-055b452cb140 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:14.383542 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-a449d127-1fbb-4c2b-9a12-055b452cb140 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018097 Jul 31 01:01:14.384451 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 23/45] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Jul 31 01:01:14 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:01:14.393984 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-1009a951-477b-48e5-a4f1-7d968fc33cd9 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:14.409344 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-1009a951-477b-48e5-a4f1-7d968fc33cd9 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018884 Jul 31 01:01:14.409870 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Jul 31 01:01:14 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:01:14.419175 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-ac257062-9b69-4dec-8c5c-e7af27df45ec admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:14.425304 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.wsgi [None req-ac257062-9b69-4dec-8c5c-e7af27df45ec admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-ac257062-9b69-4dec-8c5c-e7af27df45ec admin admin] Request method failure captured: Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Accept: application/json Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Accept-Encoding: gzip, deflate Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Connection: keep-alive Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Content-Length: 0 Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Host: 10.222.0.42 Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Openstack-Api-Version: compute 2.90 Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Openstack-System-Scope: None Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: User-Agent: python-novaclient Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Auth-Token: gAAAAABi5dRZmUbGd3OyyZpITtvY9uf7SwFOO-iOogRvdweSecf1pr3-r44qKlB8ylPhkRE8-bNp0KFrqjmq1GnDZl-xnG8NlqzDsG2Su8aSTc-xamBYUqN-wYp1FuDKKzP5LwXKVfjiwifMSgyL64ggj8haaxZpvZDzMJxgDUTmqFIKuAQkGxo Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Domain-Id: None Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Domain-Name: None Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Identity-Status: Confirmed Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Is-Admin-Project: True Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Openstack-Nova-Api-Version: 2.90 Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Domain-Id: default Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Domain-Name: Default Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Name: admin Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Role: reader,admin,member Jul 31 01:01:14.426768 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Roles: reader,admin,member Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_18e627c596bb4c99b66fda211d0e4785", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant: admin Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant-Name: admin Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User: admin Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Domain-Id: default Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Domain-Name: Default Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Id: e711c7fc76434a0b86b92dc8bcf69e87 Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Name: admin Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: method: > Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: exception: Flavor 84 could not be found. Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: action_args: {'id': '84'} Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:01:14.428337 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:14.429467 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:14.430823 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-ac257062-9b69-4dec-8c5c-e7af27df45ec admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103416) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:01:14.432047 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-ac257062-9b69-4dec-8c5c-e7af27df45ec admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012493 Jul 31 01:01:14.432047 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 24/47] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Jul 31 01:01:14 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 31 01:01:15.399485 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-1009a951-477b-48e5-a4f1-7d968fc33cd9 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:15.400507 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-1009a951-477b-48e5-a4f1-7d968fc33cd9 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001526 Jul 31 01:01:15.400906 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 24/48] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:15.919558 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-3eb81b0a-0110-4e7a-a355-439d62e9ab17 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:15.926093 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.wsgi [None req-3eb81b0a-0110-4e7a-a355-439d62e9ab17 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-3eb81b0a-0110-4e7a-a355-439d62e9ab17 admin admin] Request method failure captured: Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Accept: application/json Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Accept-Encoding: gzip, deflate Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Connection: keep-alive Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Content-Length: 0 Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Host: 10.222.0.42 Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Openstack-Api-Version: compute 2.90 Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Openstack-System-Scope: None Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: User-Agent: python-novaclient Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Auth-Token: gAAAAABi5dRbMNZJFMUuwBCSsOt6o3RvBEujbh8TEs4r7XbXyNMlu3UrUoCoxdaZmOa-b0yYYQLuiAV7XjoQ-vRLGfKbjf06rjN6GlMxrEuY-ribNc2nXhFWxQxoxyB5NfcEbCaqpL_2YaCiMzJ-NdET6wLRxoA1lmWUz0pHy1d8CzvbAlBy3AM Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Domain-Id: None Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Domain-Name: None Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Identity-Status: Confirmed Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Is-Admin-Project: True Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Openstack-Nova-Api-Version: 2.90 Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Domain-Id: default Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Domain-Name: Default Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Project-Name: admin Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Role: reader,admin,member Jul 31 01:01:15.927616 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Roles: reader,admin,member Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_18e627c596bb4c99b66fda211d0e4785", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant: admin Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-Tenant-Name: admin Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User: admin Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Domain-Id: default Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Domain-Name: Default Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Id: e711c7fc76434a0b86b92dc8bcf69e87 Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: X-User-Name: admin Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: method: > Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: exception: Flavor 451 could not be found. Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: action_args: {'id': '451'} Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:01:15.929127 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:15.930114 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: ERROR nova.api.openstack.wsgi Jul 31 01:01:15.931489 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-3eb81b0a-0110-4e7a-a355-439d62e9ab17 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103416) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:01:15.932889 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-3eb81b0a-0110-4e7a-a355-439d62e9ab17 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.390863 Jul 31 01:01:15.932889 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 25/49] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Jul 31 01:01:15 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 392 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 31 01:01:15.938479 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-c36f6fbd-fa62-4882-bfb2-c12cfa66a347 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:15.953184 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-c36f6fbd-fa62-4882-bfb2-c12cfa66a347 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018257 Jul 31 01:01:15.953741 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 25/50] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Jul 31 01:01:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:01:15.965671 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:15.988813 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.028308 Jul 31 01:01:15.989819 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 26/51] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Jul 31 01:01:15 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:01:16.000494 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-fbdddabf-7619-4ab5-a30c-1ef2154309cf admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:16.005991 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.wsgi [None req-fbdddabf-7619-4ab5-a30c-1ef2154309cf admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-fbdddabf-7619-4ab5-a30c-1ef2154309cf admin admin] Request method failure captured: Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Accept: application/json Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Accept-Encoding: gzip, deflate Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Connection: keep-alive Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Content-Length: 0 Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Host: 10.222.0.42 Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Openstack-Api-Version: compute 2.90 Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Openstack-System-Scope: None Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: User-Agent: python-novaclient Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Auth-Token: gAAAAABi5dRbMNZJFMUuwBCSsOt6o3RvBEujbh8TEs4r7XbXyNMlu3UrUoCoxdaZmOa-b0yYYQLuiAV7XjoQ-vRLGfKbjf06rjN6GlMxrEuY-ribNc2nXhFWxQxoxyB5NfcEbCaqpL_2YaCiMzJ-NdET6wLRxoA1lmWUz0pHy1d8CzvbAlBy3AM Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Domain-Id: None Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Domain-Name: None Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Identity-Status: Confirmed Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Is-Admin-Project: True Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Openstack-Nova-Api-Version: 2.90 Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Domain-Id: default Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Domain-Name: Default Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Project-Name: admin Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Role: reader,admin,member Jul 31 01:01:16.007498 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Roles: reader,admin,member Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/18e627c596bb4c99b66fda211d0e4785"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_18e627c596bb4c99b66fda211d0e4785", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant: admin Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant-Id: 18e627c596bb4c99b66fda211d0e4785 Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-Tenant-Name: admin Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User: admin Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Domain-Id: default Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Domain-Name: Default Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Id: e711c7fc76434a0b86b92dc8bcf69e87 Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: X-User-Name: admin Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: method: > Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: exception: Flavor 451 could not be found. Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: action_args: {'id': '451'} Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:01:16.009121 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:16.010244 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: ERROR nova.api.openstack.wsgi Jul 31 01:01:16.011631 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-fbdddabf-7619-4ab5-a30c-1ef2154309cf admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103417) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:01:16.012832 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-fbdddabf-7619-4ab5-a30c-1ef2154309cf admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012015 Jul 31 01:01:16.012832 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 26/52] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Jul 31 01:01:15 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 31 01:01:16.966775 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:16.967835 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001526 Jul 31 01:01:16.968231 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:17.497317 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-52dca232-d882-4cb6-a481-2878c7f2ec53 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:01:17.516176 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-52dca232-d882-4cb6-a481-2878c7f2ec53 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.409045 Jul 31 01:01:17.516719 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 27/54] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Jul 31 01:01:17 2022] POST /compute/v2.1/flavors => generated 404 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:01:18.487596 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:18.488598 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001464 Jul 31 01:01:18.488993 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 28/55] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:18 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:19.031552 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-4b2e3748-a6a5-4cc7-b80c-94ba8346de9c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:01:19.050425 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-4b2e3748-a6a5-4cc7-b80c-94ba8346de9c admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.417282 Jul 31 01:01:19.050969 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 28/56] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Jul 31 01:01:18 2022] POST /compute/v2.1/flavors => generated 405 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:01:20.029407 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:20.030410 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001426 Jul 31 01:01:20.030812 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 29/57] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:20.558492 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-b3a46dbf-93d3-40b7-90d9-3339a799d239 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:01:20.577518 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-b3a46dbf-93d3-40b7-90d9-3339a799d239 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.405615 Jul 31 01:01:20.578068 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 29/58] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Jul 31 01:01:20 2022] POST /compute/v2.1/flavors => generated 407 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:01:21.546322 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:21.547321 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Jul 31 01:01:21.547880 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 30/59] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:22.081524 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-27610126-0921-491f-9ad4-63662408f7e6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:01:22.100126 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-27610126-0921-491f-9ad4-63662408f7e6 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.414888 Jul 31 01:01:22.100664 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 30/60] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Jul 31 01:01:21 2022] POST /compute/v2.1/flavors => generated 409 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:01:23.083009 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:23.084045 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Jul 31 01:01:23.084443 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 31/61] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:01:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:01:23.607777 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-dfa313d5-88ff-42de-a341-aff39a046f82 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:01:23.627780 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-dfa313d5-88ff-42de-a341-aff39a046f82 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.403536 Jul 31 01:01:23.628319 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 31/62] 10.222.0.42 () {60 vars in 1202 bytes} [Sun Jul 31 01:01:23 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:02:15.670228 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:02:15.671216 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-88865b7a-ea6d-4f3f-ab80-3a878e035852 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001416 Jul 31 01:02:15.671715 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 32/63] 10.222.0.42 () {54 vars in 930 bytes} [Sun Jul 31 01:02:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:02:15.682948 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:02:16.066496 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-d32560a2-afe7-4d9e-a31d-2f0b0ed3b3f9 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:02:16.087102 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-d32560a2-afe7-4d9e-a31d-2f0b0ed3b3f9 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.408684 Jul 31 01:02:16.087660 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 32/64] 10.222.0.42 () {60 vars in 1291 bytes} [Sun Jul 31 01:02:15 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:02:16.108268 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG dbcounter [-] [103416] Writing DB stats nova_api:SELECT=19,nova_api:SAVEPOINT=1,nova_api:INSERT=1,nova_api:RELEASE=1 {{(pid=103416) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:02:16.112336 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-a033cb7e-e001-4591-84b2-3306e88c006f admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:02:16.120177 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-a033cb7e-e001-4591-84b2-3306e88c006f admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013252 Jul 31 01:02:16.120691 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 33/65] 10.222.0.42 () {60 vars in 1282 bytes} [Sun Jul 31 01:02:16 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:02:16.134086 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-63e45f77-f68f-432e-a083-026012679bfd admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:02:16.141823 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-63e45f77-f68f-432e-a083-026012679bfd admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013258 Jul 31 01:02:16.142345 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 33/66] 10.222.0.42 () {60 vars in 1285 bytes} [Sun Jul 31 01:02:16 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:02:16.155078 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:02:16.164729 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013532 Jul 31 01:02:16.165305 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 34/67] 10.222.0.42 () {60 vars in 1285 bytes} [Sun Jul 31 01:02:16 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:02:16.177611 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-e7ecbec0-49d4-4f01-92e2-7d24c1fbe31d admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:02:16.185302 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-e7ecbec0-49d4-4f01-92e2-7d24c1fbe31d admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011604 Jul 31 01:02:16.185829 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 34/68] 10.222.0.42 () {60 vars in 1282 bytes} [Sun Jul 31 01:02:16 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:05:50.166700 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:05:50.168640 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002640 Jul 31 01:05:50.169353 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 35/69] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:05:50 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:05:50.313345 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_api:SELECT=6 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:05:50.582218 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-62cd60c9-98f8-4426-856e-bcd1ec47c844 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:05:50.587094 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:05:50.587094 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: warnings.warn(msg) Jul 31 01:05:50.589515 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-62cd60c9-98f8-4426-856e-bcd1ec47c844 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:05:50.590286 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-62cd60c9-98f8-4426-856e-bcd1ec47c844 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:05:50.591731 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-62cd60c9-98f8-4426-856e-bcd1ec47c844 admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:05:50.592494 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-62cd60c9-98f8-4426-856e-bcd1ec47c844 admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:05:50.654815 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-62cd60c9-98f8-4426-856e-bcd1ec47c844 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.342538 Jul 31 01:05:50.655865 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 35/70] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Jul 31 01:05:50 2022] GET /compute/v2.1/os-services => generated 791 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:05:51.550835 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:05:51.552555 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002393 Jul 31 01:05:51.553220 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 36/71] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:05:51 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:05:51.776294 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-8fa47a67-6b99-4cec-925b-2bd295b0b5fe admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:05:51.778710 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:05:51.778710 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: warnings.warn(msg) Jul 31 01:05:51.779961 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8fa47a67-6b99-4cec-925b-2bd295b0b5fe admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:05:51.780406 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8fa47a67-6b99-4cec-925b-2bd295b0b5fe admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:05:51.781252 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8fa47a67-6b99-4cec-925b-2bd295b0b5fe admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:05:51.781678 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-8fa47a67-6b99-4cec-925b-2bd295b0b5fe admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:05:51.801974 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-8fa47a67-6b99-4cec-925b-2bd295b0b5fe admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.081842 Jul 31 01:05:51.802562 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 36/72] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Jul 31 01:05:51 2022] GET /compute/v2.1/os-services => generated 791 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:06:03.109070 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:06:03.110185 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001563 Jul 31 01:06:03.110597 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:06:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:06:03.383201 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_cell0:SELECT=4 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:06:03.383888 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_cell1:SELECT=4 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:06:03.384506 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG dbcounter [-] [103417] Writing DB stats nova_api:SELECT=8 {{(pid=103417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:06:03.438728 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-90a8c0d3-9692-4b0b-976a-c7bac4b48f30 admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:06:03.441359 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:06:03.441359 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: warnings.warn(msg) Jul 31 01:06:03.442701 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-90a8c0d3-9692-4b0b-976a-c7bac4b48f30 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:06:03.443139 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-90a8c0d3-9692-4b0b-976a-c7bac4b48f30 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:06:03.444535 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-90a8c0d3-9692-4b0b-976a-c7bac4b48f30 admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:06:03.444951 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-90a8c0d3-9692-4b0b-976a-c7bac4b48f30 admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:06:03.471548 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-90a8c0d3-9692-4b0b-976a-c7bac4b48f30 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.093372 Jul 31 01:06:03.472295 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 37/74] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Jul 31 01:06:03 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:06:04.359678 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: DEBUG nova.api.openstack.wsgi [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] Calling method '>' {{(pid=103416) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:06:04.360744 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: INFO nova.api.openstack.requestlog [None req-cf0695c4-df53-4f11-b83a-b39c11ebdc1e admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001529 Jul 31 01:06:04.361318 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: [pid: 103416|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Jul 31 01:06:04 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:06:04.561205 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG nova.api.openstack.wsgi [None req-e0631bc3-2603-437a-8a34-4e7a4573ff7e admin admin] Calling method '>' {{(pid=103417) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:06:04.563657 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 01:06:04.563657 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: warnings.warn(msg) Jul 31 01:06:04.565007 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-e0631bc3-2603-437a-8a34-4e7a4573ff7e admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:06:04.565435 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-e0631bc3-2603-437a-8a34-4e7a4573ff7e admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:06:04.566213 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-e0631bc3-2603-437a-8a34-4e7a4573ff7e admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:06:04.566659 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: DEBUG oslo_concurrency.lockutils [None req-e0631bc3-2603-437a-8a34-4e7a4573ff7e admin admin] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103417) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:06:04.597439 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: INFO nova.api.openstack.requestlog [None req-e0631bc3-2603-437a-8a34-4e7a4573ff7e admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.094408 Jul 31 01:06:04.598028 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: [pid: 103417|app: 0|req: 38/76] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Jul 31 01:06:04 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:07:04.058761 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: Sun Jul 31 01:07:04 2022 - graceful shutdown triggered... Jul 31 01:07:04.059814 dv-cinder-master-3-iscsi devstack@n-api.service[103416]: Gracefully killing worker 1 (pid: 103416)... Jul 31 01:07:04.060424 dv-cinder-master-3-iscsi devstack@n-api.service[103417]: Gracefully killing worker 2 (pid: 103417)... Jul 31 01:07:04.059974 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Jul 31 01:07:05.059726 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: worker 1 buried after 1 seconds Jul 31 01:07:05.060487 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: worker 2 buried after 1 seconds Jul 31 01:07:05.060487 dv-cinder-master-3-iscsi devstack@n-api.service[103415]: goodbye to uWSGI. Jul 31 01:07:05.061633 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Jul 31 01:07:05.062269 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Jul 31 01:07:05.065682 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 31 01:07:05.078652 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 31 01:07:05 2022] *** Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: nodename: dv-cinder-master-3-iscsi Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: machine: x86_64 Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: clock source: unix Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: pcre jit disabled Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: detected number of CPU cores: 4 Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: current working directory: / Jul 31 01:07:05.081768 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: detected binary path: /usr/bin/uwsgi-core Jul 31 01:07:05.083101 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: your processes number limit is 47680 Jul 31 01:07:05.083101 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: your memory page size is 4096 bytes Jul 31 01:07:05.083101 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: detected max file descriptor number: 2048 Jul 31 01:07:05.083101 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: lock engine: pthread robust mutexes Jul 31 01:07:05.083101 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: thunder lock: enabled Jul 31 01:07:05.083101 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 31 01:07:05.083101 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 31 01:07:05.126609 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: Python main interpreter initialized at 0x5612a9fbd9c0 Jul 31 01:07:05.126609 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: python threads support enabled Jul 31 01:07:05.126609 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: your server socket listen backlog is limited to 100 connections Jul 31 01:07:05.126609 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: your mercy for graceful operations on workers is 90 seconds Jul 31 01:07:05.127331 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: mapped 403077 bytes (393 KB) for 2 cores Jul 31 01:07:05.127331 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: *** Operational MODE: preforking *** Jul 31 01:07:05.131404 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: *** uWSGI is running in multiple interpreter mode *** Jul 31 01:07:05.131404 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: spawned uWSGI master process (pid: 117928) Jul 31 01:07:05.131404 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: spawned uWSGI worker 1 (pid: 117942, cores: 1) Jul 31 01:07:05.131404 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: spawned uWSGI worker 2 (pid: 117943, cores: 1) Jul 31 01:07:05.131404 dv-cinder-master-3-iscsi devstack@n-api.service[117928]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 31 01:07:05.127779 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 31 01:07:05.844744 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 31 01:07:06.052035 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 31 01:07:07.091055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 01:07:07.091055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: from cryptography.utils import int_from_bytes Jul 31 01:07:07.091829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 01:07:07.091829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: from cryptography.utils import int_from_bytes Jul 31 01:07:07.302705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 01:07:07.302705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: from cryptography.utils import int_from_bytes Jul 31 01:07:07.303727 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 01:07:07.303727 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: from cryptography.utils import int_from_bytes Jul 31 01:07:09.732411 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 01:07:09.733469 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 01:07:09.733469 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 01:07:09.733810 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 01:07:09.733810 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 01:07:09.734496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.734496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.734935 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.735140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.735509 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.735831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.736161 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.736488 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.736825 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.737093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.737403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.737738 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.738067 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.738383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.738700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.739093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.739403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.739732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.740044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.740558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.742184 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.742184 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.742184 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.742933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.742933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.743474 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.743474 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.743961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.743961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.744495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.744749 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.745009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.745272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.746223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.746223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.746731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.746731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.747203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.747203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.747709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.747976 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.748240 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.748485 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.749560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.749560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.749560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.750350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.750350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.750350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.751131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.751131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.751131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.751831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.751831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.752301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.752538 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.752776 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.753060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.753318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.753554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.753792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.754384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.754384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.759048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.760968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.762767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.764732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.766620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.766620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.766620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.766620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.766620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.766620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.768031 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.768031 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.768031 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.768031 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.768031 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.771049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.772985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.772985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.772985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.772985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.772985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.772985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.774342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.774342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.774342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.774342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.774342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.777459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.779580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.781516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.781516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.781516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.781516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.781516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.781516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.781516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.786420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.788248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.790068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.793954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.795956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.797885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.806808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.808739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.810834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.812694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.814507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.818692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.820545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.822523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.825416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.828792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.830398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.832460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.834920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.838550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.840072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.841393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.842908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.842908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.842908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.842908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.842908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.842908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.844615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.844615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.844615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.844615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.844615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.845813 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.845813 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.845813 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.845813 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.845813 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.848713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.850301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.850301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.850301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.850301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.850301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.850301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.851720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.851720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.851720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.851720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.851720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.852880 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.852880 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.852880 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.853562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.853562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.853562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.854477 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.854477 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.854477 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.855235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.855235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.855828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.855828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.856303 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.856303 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.856773 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.856773 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.858302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.858302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.858302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.858302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.858302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.859833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.859833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.859833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.859833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.859833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.861905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.863730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.863730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.863730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.863730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.863730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.863730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.865346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.865346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.865346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.865346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.865346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.866553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.866553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.866553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.866553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.868551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.868551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.868551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.868551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.868551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.868551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.870383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.870383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.870383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.870383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.870383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.870383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.870383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.872323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.872323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.872323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.872323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.872323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.872323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.873746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.873746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.873746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.873746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.874645 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.874645 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.875624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.876325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.876984 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.877666 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.878358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.879034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.880276 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.880915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.881682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.882429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.883017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.883644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.884211 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.884798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.885399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.885990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.886566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.887137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.888186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.888775 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.889350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.889892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.890485 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.891060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.893605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.893605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.893605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.893605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.894173 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.894818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.896088 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.896651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.897213 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.897788 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.898352 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.898937 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.899548 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.900127 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.900938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.901501 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.901954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.902528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.903100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.903698 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.904277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.904892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.905463 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.906025 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.906613 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.907179 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.907769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.908381 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.909178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.909736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.910280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.911098 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.911707 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.912283 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.912818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.913398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.913977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.914546 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.915127 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.915750 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.916330 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.916922 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.917499 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.918068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.918616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.919219 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.919858 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.920455 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.921075 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.921915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.922538 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.923110 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.923729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.924288 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.924846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.925396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.925571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 01:07:09.926002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 01:07:09.926077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.926378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 01:07:09.926559 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.927098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 01:07:09.927218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.927465 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 01:07:09.927577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.927792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.928041 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.928152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.928444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.928637 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.928797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.929518 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.929709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.929892 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.930046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.930210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.930210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.930519 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.930680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.930680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.930983 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.931134 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.931284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.931499 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.931667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.932324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.932507 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.932507 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.932820 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.932971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.932971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.933265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.933416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.933817 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.934000 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.934161 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.934606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.934772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.934998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.935204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.935410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.935624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.935790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.936244 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.936400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.936635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.936910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.937185 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.937720 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.937876 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.938153 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.938325 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.938712 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.938885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.939037 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.939780 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.940005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.940005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.940323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.940480 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.940480 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.940825 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.940978 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.941143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.941306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.941456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.941679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.941845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.942947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.943119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.943273 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.943455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.943607 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.943768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.944046 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.944210 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.944370 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.944673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.944829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.944988 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.945279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.945436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.945606 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.945837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.946001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.946159 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.946479 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.946702 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.946885 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.947162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.947320 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.947320 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.947697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.947855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.948013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.948207 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.948371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.948534 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.948781 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.948947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.949138 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.949308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.949479 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.950119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.950288 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.950441 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.950441 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.950722 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.950889 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.951052 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.951213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.951400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.951556 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.951717 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.951895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.952126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.952294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.952553 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.953188 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.953358 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.953358 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.953664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.953815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.953815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.954117 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.954267 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.954434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.954595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.954744 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.955027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.955328 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.955529 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.955782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.955945 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.956103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.956266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.956417 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.956576 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.956799 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.956975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.957146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.957403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.957619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.958330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.958620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.958780 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.959564 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.960420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.960600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.960600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.960892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.961043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.961043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.961369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.961528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.961528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.961821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.961973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.961973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.962280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.962431 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.962431 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.962715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.962897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.962897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.963670 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.963823 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.963823 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.964128 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.964295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.964295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.964602 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.964763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.964763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.965835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.966003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.966003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.966327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.966491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.966491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.966852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.967020 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.967173 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.967323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.967323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.967820 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.967973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.968125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.968275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.968275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.968587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.968741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.968741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.968741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.968741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.969319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.969472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.969472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.969774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.969939 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.969939 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.970742 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.970905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.970905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.971224 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.971407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.971564 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.971752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.971752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.971752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.972288 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.972441 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.972591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.972752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 01:07:09.972752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.973068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.973642 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.973642 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.973953 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.974114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.974114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.974412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.974563 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.974720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.974880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.975032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.975180 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.975338 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.975731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.975884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.976046 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.976197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.976346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.976498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.976815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.977043 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.977248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.977462 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.977641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.977904 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.978085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.978246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.978562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.978720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.978908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.979202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.979383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.979611 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.979904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.980073 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.980853 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.981037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.981365 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.981365 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.981688 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.981840 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.981840 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.982143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.982304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.982304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.982613 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.982774 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.982925 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.983081 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.983241 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.983450 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.983878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.983878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.984182 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.984364 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.984606 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.984762 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.984921 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.985196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.985354 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.985515 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.985803 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.985985 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.986171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.986336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.986489 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.986802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.986969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.987178 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.987433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.987597 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.987843 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.988037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.988199 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.988473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.988651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.988837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.989122 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.989390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.989555 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.989737 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.989993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.990151 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.990368 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.990591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.991086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.991086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.991433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.991606 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.991771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.991962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.992207 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.992365 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.992581 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.992736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.992907 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.993197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.993353 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.993511 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.993820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.994088 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.994262 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.994465 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.994629 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.994802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.995067 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.995252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.995571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.995729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.995952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.996106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.996253 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.996528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.996691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.996862 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.997031 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.997241 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.997405 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.997577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.997880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.998085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.998300 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.998515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.998705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.998873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.999097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.999272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.999473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.999744 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:09.999923 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.000121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.000325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.000491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.000736 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.000911 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.001129 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.001339 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.001469 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.001667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.001955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.002115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.002283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.002561 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.002871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.003105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.003277 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.003492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.003815 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.004035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.004252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.004944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.005116 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.005524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.005904 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.006108 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.006246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.006515 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.006744 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.007003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.007197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.007567 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.007929 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.008351 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.008520 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.008673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.008850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.008992 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.009169 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.009486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.009601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.009802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.010064 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.010213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.010433 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.010635 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.010790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.011063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.011222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.011438 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.011722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.011882 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.012034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.012345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.012680 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.012835 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.013032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.013272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.013514 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.013858 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.014015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.014188 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.014670 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.014834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.014982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.015333 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.015674 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.015674 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.016147 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.016416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.016416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.016734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.016877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.017090 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.017245 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.017420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.017689 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.017906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.018072 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.018413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.018738 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.018915 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.019121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.019269 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.019502 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.019686 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.019971 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.020283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.020989 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.021142 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.021142 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.021424 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.021597 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.021764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.021927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.022082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.022223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.022378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.022509 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.022691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.022828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.023007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.023160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.023316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.023531 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.023686 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.023837 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.023991 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.024141 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.024309 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.024465 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.024616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.024778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.024944 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.025104 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.025270 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.025427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.025647 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.025864 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.026060 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.026258 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.026455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.026648 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.026841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.027002 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.027159 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.027311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.027610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.027784 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.027946 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.028049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.028161 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.028324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.028493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.028493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.028866 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029182 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029650 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029650 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.029979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.030281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.030437 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.030600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.030755 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.030941 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.031098 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.031269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.031453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.031614 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.031751 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.031929 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.032098 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.032256 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.032417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.032583 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.032748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.032928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.033095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.033249 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.033417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.033570 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.033736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.034000 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.034174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.034348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.034506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.034675 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.034845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.035202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.035451 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.035597 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.035775 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.035911 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.036059 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.036276 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.036464 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.036656 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.036846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.037057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.037252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.037426 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.037582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.037749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.037920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.038093 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.038252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.038252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.038537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.038694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.038855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.039083 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.039317 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.039555 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.039732 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.039889 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.040046 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.040220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.040377 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.040549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.040719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.040879 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.041036 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.041194 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.041346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.041499 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.041679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.041830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.041991 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.042151 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.042303 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.042462 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.042631 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.042875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.043084 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.043237 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.043491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.043677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.043886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.044098 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.044298 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.044496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.044690 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.044897 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.045117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.045272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.045423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.045559 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.045696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.045843 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.045980 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046401 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046799 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046799 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046799 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.046799 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.047298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.047460 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.047596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.047752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.047900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.048035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.048181 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.048330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.048466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.048610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.048745 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.048882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.049033 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.049176 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.049320 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.049472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.049620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.049768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.049939 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.050088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.050225 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.050345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.050482 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.050638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.050774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.050919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.051063 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.051207 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.051434 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.051434 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.051434 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.051434 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.051947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052361 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052498 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052649 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052788 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.052944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053031 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053484 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.053921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.054061 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.054233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.054381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.054533 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.054680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.054828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.055042 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.055190 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.055335 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.055518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.055664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.055823 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.056050 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.056366 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.056825 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.056995 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.057146 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.057294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.057430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.057567 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.057719 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.057872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.058010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.058149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.058288 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.058429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.058574 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.058721 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.058859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.059062 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.059213 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.059707 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.059878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.060052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.060218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.060357 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.060495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.060646 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.060793 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.060937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.061084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.061221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.061366 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.061522 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.061687 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.061825 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.061973 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.062107 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.062249 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.062500 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.062634 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.062769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.062904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.063052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.063207 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.063399 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.063573 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.063734 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.063901 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.064057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.064219 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.064378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.064532 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.064685 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.064833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.064987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.065135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.065301 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.065454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.065610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.065772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.065924 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.066103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.066262 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.066426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.066599 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.066755 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.067554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.067726 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.067895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.068065 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.068235 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.068389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.068559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.068718 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.068870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.069036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.069201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.069369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.069543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.069709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117943) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 01:07:10.069866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.069866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.070159 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.070402 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.070728 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.071059 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.071542 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.071729 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.072046 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.072357 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.072681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.073006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.073327 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.073641 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.073955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.074285 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.074601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.074918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.075247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.075631 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.075970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.076285 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.076602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.076918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.077235 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.077552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.077870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.078189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.078507 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.078833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.079153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.079492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.079810 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.080128 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.080444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.080756 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.081092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.081408 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.081798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.082032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.082345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.083228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.083575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.084026 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.084342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.084658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.084970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.085272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.085608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.085930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.086250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.086565 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.087003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.087339 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.087682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.087999 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.088314 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.088633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.088953 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.089290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.089605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.089919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.090232 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.090566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.090880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.091211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.091680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.092111 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.092324 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.092733 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.093052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.093413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.093884 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.094224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.094561 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.094902 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.095232 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.095585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.095926 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.096265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.096582 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.096929 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.097244 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.097576 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.097910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.098270 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.098623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.099063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.099431 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.099758 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.100097 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.100431 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.100746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.101060 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.101380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.101696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.102013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.102346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.102658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.102979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.103298 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.103677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.104005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.104271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.104601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.104913 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.105286 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.105592 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.106210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.106557 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.106870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.107198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.107573 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.107889 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.108224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.108538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.108877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.109244 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.109561 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.109890 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.110253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.110585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.110942 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.111286 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.111686 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.111975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.112311 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.112755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.112946 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.113268 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.113585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.113934 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.114248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.114568 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.114880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.115193 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.115531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.115877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.116380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.116715 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.117039 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.118595 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.120665 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.121065 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.121487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.121831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.123526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.123919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.124278 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.124732 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.125071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.125419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.125756 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.126089 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.126426 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.126760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.127090 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.127456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.127792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.128119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.128424 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.128755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.129090 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.130248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.130598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.130931 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.131266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.131654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.131950 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.132439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.132772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.133110 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.133435 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.133766 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.134139 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.134477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.134822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.135179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.135556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.135887 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.136220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.136552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.136940 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.137268 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.137593 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.137922 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.138254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.138580 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.138905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.139229 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.139614 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.139942 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.140266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.140591 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.142137 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.142483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.142941 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.143231 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.143605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.143962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.144550 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.144994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.146092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.146513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.146845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.147161 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.147539 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.147877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.148220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.148568 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.148887 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.149208 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.149537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.149860 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.150180 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.150524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.150853 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.151175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.151543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.151867 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.152484 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.152849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.153188 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.153533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.153868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.154205 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.154528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.154865 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.155184 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.155540 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.155872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.156196 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.156514 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.156830 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.157165 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.157509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.157825 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.158149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.158464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.158808 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.159127 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.159473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.159790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.160114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.160427 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.160784 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.161103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.161398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.161731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.162048 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.162477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.162661 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.162990 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.163312 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.163692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.164044 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.164389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.164705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.165020 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.165354 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.165791 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.166127 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.166450 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.166771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.167140 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.167618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.168130 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.168483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.168819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.169172 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.169520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.169840 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.170388 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.170577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.170850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.171152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.171523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.172084 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.172465 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.172798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.173128 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.173466 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.173795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.174121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.174446 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.174771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.175094 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.175443 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.175769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.176105 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.176510 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.176845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.177176 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.177562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.179504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.180709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.181747 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.182873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.184851 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.186322 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.187382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.188316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.189254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.190403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.191468 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.193157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.194859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.195872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.197321 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.198278 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.199203 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.200106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.200917 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.201872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.202936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.204497 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.205384 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.206220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.207577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.208953 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.209620 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.210299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.210973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.211670 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.212311 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.221879 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.226855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.227961 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.229762 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.230404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.232253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.232845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.233438 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.233952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.239892 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.241173 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.242206 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.243144 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.244197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.246126 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.247093 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.248336 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.249475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.250469 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.251461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.252528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.253512 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.255472 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO dbcounter [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Registered counter for database nova_api Jul 31 01:07:10.255915 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.256597 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.257218 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.257843 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.258912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.259593 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.260195 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.260832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.261427 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.262058 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.262657 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.263248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.263866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.264470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.265094 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.266099 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.266705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.267343 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.267981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.268858 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.269463 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.270142 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.272471 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.272679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.273070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.273586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.273776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.274282 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.274453 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.278833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.280437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 01:07:10.281411 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=117942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 01:07:10.281550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_db.sqlalchemy.engines [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117943) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.294156 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writer thread running {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.344138 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO dbcounter [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Registered counter for database nova_api Jul 31 01:07:10.368284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_db.sqlalchemy.engines [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117942) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.376778 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writer thread running {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.413251 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.context [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d(cell1) {{(pid=117943) load_cells /opt/stack/nova/nova/context.py:464}} Jul 31 01:07:10.421598 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:07:10.425057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:07:10.428628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:07:10.430306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:07:10.434086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO dbcounter [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Registered counter for database nova_cell0 Jul 31 01:07:10.440414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.context [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d(cell1) {{(pid=117942) load_cells /opt/stack/nova/nova/context.py:464}} Jul 31 01:07:10.443492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO dbcounter [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Registered counter for database nova_cell1 Jul 31 01:07:10.445433 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:07:10.447490 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:07:10.448363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:07:10.450472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:07:10.452701 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO dbcounter [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Registered counter for database nova_cell0 Jul 31 01:07:10.455587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_db.sqlalchemy.engines [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117943) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.457352 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO dbcounter [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Registered counter for database nova_cell1 Jul 31 01:07:10.458082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_db.sqlalchemy.engines [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117943) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.466053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_db.sqlalchemy.engines [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117942) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.467904 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_db.sqlalchemy.engines [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117942) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.470653 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writer thread running {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.473303 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writer thread running {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.474695 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writer thread running {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.479707 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writer thread running {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.639475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO dbcounter [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Registered counter for database nova_cell0 Jul 31 01:07:10.647429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_db.sqlalchemy.engines [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117942) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.652567 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writer thread running {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.762039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO dbcounter [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Registered counter for database nova_cell0 Jul 31 01:07:10.772038 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_db.sqlalchemy.engines [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=117943) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 01:07:10.780026 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writer thread running {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 01:07:10.981311 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: WARNING keystonemiddleware.auth_token [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 01:07:10.988987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: WARNING keystonemiddleware.auth_token [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 01:07:11.403925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: WARNING keystonemiddleware.auth_token [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 01:07:11.409400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: WARNING keystonemiddleware.auth_token [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 01:07:11.415795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: WARNING keystonemiddleware.auth_token [None req-0c67287f-8aa8-4fb3-bcba-674be49b2f05 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 01:07:11.418377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: WARNING keystonemiddleware.auth_token [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 01:07:11.422983 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5612a9fbd9c0 pid: 117942 (default app) Jul 31 01:07:11.857047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: WARNING keystonemiddleware.auth_token [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 31 01:07:11.863255 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: WARNING keystonemiddleware.auth_token [None req-d6daa882-f3ba-4862-9e72-b34b0d9da242 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 31 01:07:11.869439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5612a9fbd9c0 pid: 117943 (default app) Jul 31 01:08:00.649234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:00.652198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:00.655813 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:00.657012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:03.503989 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5d633919-1be9-4ace-9b3c-ed0a3087baaf tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Action: 'create', calling method: .version_select at 0x7f3e8177bee0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-662250161", "metadata": {"Type": "work"}}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5d633919-1be9-4ace-9b3c-ed0a3087baaf tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Request method failure captured: Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 121 Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dXwsECwxLpjfOJ82rg9-8-CiCZofBousUu4iq9Z8hI9dyLpgbX0BxF8RaegcFkSAji4DvVwcDQysmQBVhShN_-SlTmsiQv3gZ6PTX7DR8wUqGC-4XRDfWi6y5gmvg7tWTE49VVdW4ll8i_fIiziuWebfh6GEHWluvotyOZuE_AGpHM Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:08:03.535224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/be70baea277b4b34b677053f5810c337"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_be70baea277b4b34b677053f5810c337", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 3c537a32dd314556a29e7f3b47f0a874 Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-662250161", "metadata": {"Type": "work"}}} Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: Jul 31 01:08:03.536764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-662250161', 'metadata': {'Type': 'work'}}}} Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise error Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi '#$%' Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:08:03.537589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 31 01:08:03.538938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 31 01:08:03.540124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 31 01:08:03.540124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:03.540124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5d633919-1be9-4ace-9b3c-ed0a3087baaf tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:08:03.540124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5d633919-1be9-4ace-9b3c-ed0a3087baaf tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 2.908777 Jul 31 01:08:03.540124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 1/1] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Jul 31 01:08:00 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 2911 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 31 01:08:04.155835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-65fb6558-e5f7-4920-ad46-0f6fd8229a00 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Action: 'create', calling method: .version_select at 0x7f3e8180d940>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1216735760", "metadata": {"Type": "work"}}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-65fb6558-e5f7-4920-ad46-0f6fd8229a00 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Request method failure captured: Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 120 Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dXwsECwxLpjfOJ82rg9-8-CiCZofBousUu4iq9Z8hI9dyLpgbX0BxF8RaegcFkSAji4DvVwcDQysmQBVhShN_-SlTmsiQv3gZ6PTX7DR8wUqGC-4XRDfWi6y5gmvg7tWTE49VVdW4ll8i_fIiziuWebfh6GEHWluvotyOZuE_AGpHM Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:08:04.163676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/be70baea277b4b34b677053f5810c337"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_be70baea277b4b34b677053f5810c337", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 3c537a32dd314556a29e7f3b47f0a874 Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1216735760", "metadata": {"Type": "work"}}} Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: Jul 31 01:08:04.165034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1216735760', 'metadata': {'Type': 'work'}}}} Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise error Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi '0' Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:08:04.165807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 31 01:08:04.167056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 31 01:08:04.168205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 31 01:08:04.168205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.168205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-65fb6558-e5f7-4920-ad46-0f6fd8229a00 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:08:04.168205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-65fb6558-e5f7-4920-ad46-0f6fd8229a00 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.620932 Jul 31 01:08:04.172168 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 1/2] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Jul 31 01:08:03 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 628 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 31 01:08:04.184776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0928d87d-1543-4837-bf80-7b343e0f1393 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Action: 'create', calling method: .version_select at 0x7f3e81680670>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-845775534", "metadata": {"Type": "work"}}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0928d87d-1543-4837-bf80-7b343e0f1393 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Request method failure captured: Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 118 Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dXwsECwxLpjfOJ82rg9-8-CiCZofBousUu4iq9Z8hI9dyLpgbX0BxF8RaegcFkSAji4DvVwcDQysmQBVhShN_-SlTmsiQv3gZ6PTX7DR8wUqGC-4XRDfWi6y5gmvg7tWTE49VVdW4ll8i_fIiziuWebfh6GEHWluvotyOZuE_AGpHM Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:08:04.201382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/be70baea277b4b34b677053f5810c337"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_be70baea277b4b34b677053f5810c337", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 3c537a32dd314556a29e7f3b47f0a874 Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-845775534", "metadata": {"Type": "work"}}} Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: Jul 31 01:08:04.202949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-845775534', 'metadata': {'Type': 'work'}}}} Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise error Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi '' Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:08:04.203901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 31 01:08:04.205175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 31 01:08:04.206376 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 31 01:08:04.206376 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:04.206376 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0928d87d-1543-4837-bf80-7b343e0f1393 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:08:04.207041 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0928d87d-1543-4837-bf80-7b343e0f1393 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.034208 Jul 31 01:08:04.213527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 2/3] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Jul 31 01:08:04 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 42 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 31 01:08:04.222326 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:04.224525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:04.225491 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:04.227965 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:04.239809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-21888550-7032-4d07-9d1e-08f07dde8431 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Calling method '.version_select at 0x7f3e8179cca0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:05.136932 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-21888550-7032-4d07-9d1e-08f07dde8431 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] HTTP exception thrown: Volume tempest-invalid-1178997898 could not be found. Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-21888550-7032-4d07-9d1e-08f07dde8431 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Request method failure captured: Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1178997898 HTTP/1.1 Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dXwsECwxLpjfOJ82rg9-8-CiCZofBousUu4iq9Z8hI9dyLpgbX0BxF8RaegcFkSAji4DvVwcDQysmQBVhShN_-SlTmsiQv3gZ6PTX7DR8wUqGC-4XRDfWi6y5gmvg7tWTE49VVdW4ll8i_fIiziuWebfh6GEHWluvotyOZuE_AGpHM Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:08:05.143008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/be70baea277b4b34b677053f5810c337"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_be70baea277b4b34b677053f5810c337", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 3c537a32dd314556a29e7f3b47f0a874 Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Volume tempest-invalid-1178997898 could not be found. Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'tempest-invalid-1178997898'} Jul 31 01:08:05.144741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1178997898 could not be found. (HTTP 404) (Request-ID: req-6f6b58cf-e555-4715-99c0-1209f9b6ad3d) Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:05.145365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 31 01:08:05.146265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1178997898 could not be found. Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:08:05.147163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1178997898 could not be found. Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1178997898 could not be found. Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-21888550-7032-4d07-9d1e-08f07dde8431 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Returning 404 to user: Volume tempest-invalid-1178997898 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:08:05.150022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-21888550-7032-4d07-9d1e-08f07dde8431 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1178997898" status: 404 len: 99 microversion: 2.1 time: 0.928384 Jul 31 01:08:05.151503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Jul 31 01:08:04 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1178997898 => generated 99 bytes in 931 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 31 01:08:05.161831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ed347944-857b-4afa-9d3c-11e83bdbc365 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004818 Jul 31 01:08:05.162633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 3/5] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Jul 31 01:08:05 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 31 01:08:05.173214 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-fff72430-0ff1-4a7c-b3e2-840da3df3539 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005642 Jul 31 01:08:05.180160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 3/6] 10.222.0.42 () {58 vars in 1137 bytes} [Sun Jul 31 01:08:05 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 13 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 31 01:08:05.186553 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-35cb6a92-965e-484f-b11e-2fd3b1fd502e tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Calling method '.version_select at 0x7f3e8164ed30>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:06.343770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-35cb6a92-965e-484f-b11e-2fd3b1fd502e tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] HTTP exception thrown: Volume 23d5648c-a804-42c7-9ef7-d230395cf227 could not be found. Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-35cb6a92-965e-484f-b11e-2fd3b1fd502e tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Request method failure captured: Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/os-volumes/23d5648c-a804-42c7-9ef7-d230395cf227 HTTP/1.1 Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dXwsECwxLpjfOJ82rg9-8-CiCZofBousUu4iq9Z8hI9dyLpgbX0BxF8RaegcFkSAji4DvVwcDQysmQBVhShN_-SlTmsiQv3gZ6PTX7DR8wUqGC-4XRDfWi6y5gmvg7tWTE49VVdW4ll8i_fIiziuWebfh6GEHWluvotyOZuE_AGpHM Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:08:06.352119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/be70baea277b4b34b677053f5810c337"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_be70baea277b4b34b677053f5810c337", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 3c537a32dd314556a29e7f3b47f0a874 Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Volume 23d5648c-a804-42c7-9ef7-d230395cf227 could not be found. Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '23d5648c-a804-42c7-9ef7-d230395cf227'} Jul 31 01:08:06.353211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 23d5648c-a804-42c7-9ef7-d230395cf227 could not be found. (HTTP 404) (Request-ID: req-186577c3-9b98-43f2-bbd3-5abe1641f900) Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:06.353809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 31 01:08:06.354678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 23d5648c-a804-42c7-9ef7-d230395cf227 could not be found. Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 31 01:08:06.355559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 23d5648c-a804-42c7-9ef7-d230395cf227 could not be found. Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 23d5648c-a804-42c7-9ef7-d230395cf227 could not be found. Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-35cb6a92-965e-484f-b11e-2fd3b1fd502e tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Returning 404 to user: Volume 23d5648c-a804-42c7-9ef7-d230395cf227 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:08:06.357183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-35cb6a92-965e-484f-b11e-2fd3b1fd502e tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/23d5648c-a804-42c7-9ef7-d230395cf227" status: 404 len: 109 microversion: 2.1 time: 1.175011 Jul 31 01:08:06.358935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:08:05 2022] DELETE /compute/v2.1/os-volumes/23d5648c-a804-42c7-9ef7-d230395cf227 => generated 109 bytes in 1180 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:08:06.368461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e4355936-2581-4551-a5e8-97bcb79b2a79 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Calling method '.version_select at 0x7f3e8153a9d0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:06.426206 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-e4355936-2581-4551-a5e8-97bcb79b2a79 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] HTTP exception thrown: Volume d9bb5b82-df13-41ec-b599-90c3d1640a74 could not be found. Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e4355936-2581-4551-a5e8-97bcb79b2a79 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Request method failure captured: Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/os-volumes/d9bb5b82-df13-41ec-b599-90c3d1640a74 HTTP/1.1 Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dXwsECwxLpjfOJ82rg9-8-CiCZofBousUu4iq9Z8hI9dyLpgbX0BxF8RaegcFkSAji4DvVwcDQysmQBVhShN_-SlTmsiQv3gZ6PTX7DR8wUqGC-4XRDfWi6y5gmvg7tWTE49VVdW4ll8i_fIiziuWebfh6GEHWluvotyOZuE_AGpHM Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:08:06.429436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/be70baea277b4b34b677053f5810c337"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/be70baea277b4b34b677053f5810c337"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_be70baea277b4b34b677053f5810c337", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: be70baea277b4b34b677053f5810c337 Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesNegativeTest-1237647687 Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 3c537a32dd314556a29e7f3b47f0a874 Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesNegativeTest-1237647687-project Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Volume d9bb5b82-df13-41ec-b599-90c3d1640a74 could not be found. Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'd9bb5b82-df13-41ec-b599-90c3d1640a74'} Jul 31 01:08:06.430674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d9bb5b82-df13-41ec-b599-90c3d1640a74 could not be found. (HTTP 404) (Request-ID: req-a098fef4-34c3-419f-916a-e2fae61563f1) Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:06.431263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:08:06.432163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d9bb5b82-df13-41ec-b599-90c3d1640a74 could not be found. Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 31 01:08:06.433021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d9bb5b82-df13-41ec-b599-90c3d1640a74 could not be found. Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d9bb5b82-df13-41ec-b599-90c3d1640a74 could not be found. Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e4355936-2581-4551-a5e8-97bcb79b2a79 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] Returning 404 to user: Volume d9bb5b82-df13-41ec-b599-90c3d1640a74 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:08:06.433850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e4355936-2581-4551-a5e8-97bcb79b2a79 tempest-VolumesNegativeTest-1237647687 tempest-VolumesNegativeTest-1237647687-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/d9bb5b82-df13-41ec-b599-90c3d1640a74" status: 404 len: 109 microversion: 2.1 time: 0.068061 Jul 31 01:08:06.443077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:08:06 2022] GET /compute/v2.1/os-volumes/d9bb5b82-df13-41ec-b599-90c3d1640a74 => generated 109 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:08:06.647634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e0a40b44-ea15-44c6-8d35-3d22aa0ede95 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e81582940>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:06.780103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e0a40b44-ea15-44c6-8d35-3d22aa0ede95 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/945253c6-6d6c-413d-984e-a08b39c076d5" status: 200 len: 350 microversion: 2.1 time: 0.133990 Jul 31 01:08:06.802682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 5/9] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:08:06 2022] GET /compute/v2.1/os-volumes/945253c6-6d6c-413d-984e-a08b39c076d5 => generated 350 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:09.762009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7c98f680-08c3-458a-9f76-d41c56d2e6c3 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e8153a3a0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:09.872502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7c98f680-08c3-458a-9f76-d41c56d2e6c3 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/6032a624-0a26-4490-a99b-1679f3b8f2d5" status: 200 len: 350 microversion: 2.1 time: 0.116596 Jul 31 01:08:09.873166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 5/10] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:08:09 2022] GET /compute/v2.1/os-volumes/6032a624-0a26-4490-a99b-1679f3b8f2d5 => generated 350 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:12.976232 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1640747225", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "networks": [{"uuid": "e59245a7-bb2b-4ccb-b19c-666c6ac4de9d"}]}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:13.340046 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] validate_networks() for [('e59245a7-bb2b-4ccb-b19c-666c6ac4de9d', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:08:17.843525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:08:17.845585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:08:17.847011 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:08:17.848335 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:08:17.850575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:08:17.940680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:17.941797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:17.943727 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:17.959264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:18.100981 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b2e0bff0-588c-4138-8f33-c7f9a42a422f tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e8153a550>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:18.189695 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Getting quotas for project 66c7f61afd0d4650bd530eff18a2318b. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:08:18.231872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Getting quotas for user c4a002047e3e42338e4f57ba49e877fb and project 66c7f61afd0d4650bd530eff18a2318b. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:08:18.279294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:08:18.283517 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:08:18.457531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-71cd8084-19a0-4107-b214-b9ff1d3b4ef9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 5.785594 Jul 31 01:08:18.458547 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 6/11] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:08:12 2022] POST /compute/v2.1/servers => generated 380 bytes in 5787 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:08:18.859608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8f9ac348-e90c-4ae2-ac5d-536e5d5af4aa tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: .version_select at 0x7f3e81498940>, body: {"keypair": {"name": "tempest-keypair-1298245109", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE0B/ahkVocE+UgWwqMsli3wl8/JyrAMGS5eX+x3Oakr03g8Vc7/LbJlZzrT74xnjSLrnYdUocXL7ugsIYXGsivsbVBvRRyVGcxhCH6Yw6D8WESXlBPcGchqTwAiE5UlBw=="}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:18.969397 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b2e0bff0-588c-4138-8f33-c7f9a42a422f tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/4c3b4ab5-26bf-4d61-83aa-7a082925c77c" status: 200 len: 349 microversion: 2.1 time: 0.877825 Jul 31 01:08:18.970340 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:08:18 2022] GET /compute/v2.1/os-volumes/4c3b4ab5-26bf-4d61-83aa-7a082925c77c => generated 349 bytes in 880 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:19.003235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-63d61cfb-e865-4988-b323-65d19c5e7072 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:19.004728 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-63d61cfb-e865-4988-b323-65d19c5e7072 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:19.071613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-8f9ac348-e90c-4ae2-ac5d-536e5d5af4aa tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:08:19.104423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-8f9ac348-e90c-4ae2-ac5d-536e5d5af4aa tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:08:19.275389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8f9ac348-e90c-4ae2-ac5d-536e5d5af4aa tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.808516 Jul 31 01:08:19.276092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 7/13] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:08:18 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 817 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:19.289888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ea460d8b-1157-4c46-9a15-37487b0f5087 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e814af5e0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:19.443812 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-63d61cfb-e865-4988-b323-65d19c5e7072 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:19.480815 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-63d61cfb-e865-4988-b323-65d19c5e7072 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1215 microversion: 2.1 time: 0.506521 Jul 31 01:08:19.482552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 7/14] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:18 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1215 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:20.130741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ea460d8b-1157-4c46-9a15-37487b0f5087 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1032 microversion: 2.1 time: 0.852158 Jul 31 01:08:20.132131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 8/15] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Jul 31 01:08:19 2022] GET /compute/v2.1/os-volumes => generated 1032 bytes in 855 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:20.156460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e0b53101-e132-4540-a250-65c663f30d58 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e814ff280>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:20.524654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-69ee7350-4f0c-46f7-bcdb-4062540faf2d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:20.525126 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-69ee7350-4f0c-46f7-bcdb-4062540faf2d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:20.762801 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e0b53101-e132-4540-a250-65c663f30d58 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 692 microversion: 2.1 time: 0.616327 Jul 31 01:08:20.763746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1149 bytes} [Sun Jul 31 01:08:20 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 692 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:20.844523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-69ee7350-4f0c-46f7-bcdb-4062540faf2d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:20.894740 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-69ee7350-4f0c-46f7-bcdb-4062540faf2d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1215 microversion: 2.1 time: 0.377600 Jul 31 01:08:20.908572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 9/17] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:20 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1215 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:20.960240 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-00155ab7-2737-4bbd-be75-e33205787435 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e80456790>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:21.357534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-73aaa064-0a8d-4b55-bd76-b1c2fa301ab0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'create', calling method: .version_select at 0x7f3e8150df70>, body: {"keypair": {"name": "tempest-keypair-460569984", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGvr4zgQ2xtiRLG9Qo0Ihis0z9j1FioEdt2Bf18gIVZmBofBQ4Fh6J7hA08zr5Q51NUtdvDyJjuWlauhLhMoTOl5mS24FAiu0K27exEVhbU8yOlNC4R/xJyWT+aAPbP/rA=="}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:21.460833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-73aaa064-0a8d-4b55-bd76-b1c2fa301ab0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for project 80ea98b340d94181b2193f4a79905988. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:08:21.481359 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-73aaa064-0a8d-4b55-bd76-b1c2fa301ab0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for user 56732ebe8ee24f3f9ce8f87746e8b161 and project 80ea98b340d94181b2193f4a79905988. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:08:21.645758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-73aaa064-0a8d-4b55-bd76-b1c2fa301ab0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.877945 Jul 31 01:08:21.651462 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 9/18] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Jul 31 01:08:20 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 884 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 31 01:08:21.954999 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:21.957424 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:21.984917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:21.985374 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:21.988109 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:21.988562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:22.223065 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-00155ab7-2737-4bbd-be75-e33205787435 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1032 microversion: 2.1 time: 1.312117 Jul 31 01:08:22.224817 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 10/19] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Jul 31 01:08:20 2022] GET /compute/v2.1/os-volumes => generated 1032 bytes in 1315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:22.245507 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-be202d6f-bf39-4392-a0e7-3ebae39076ee tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e803f3d30>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:22.572264 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:22.608853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8d41ff0e-d64d-4586-8805-bbd687cf3375 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1219 microversion: 2.1 time: 0.670604 Jul 31 01:08:22.609696 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 10/20] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:21 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1219 bytes in 672 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:22.728448 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-be202d6f-bf39-4392-a0e7-3ebae39076ee tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.490783 Jul 31 01:08:22.745422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 11/21] 10.222.0.42 () {58 vars in 1167 bytes} [Sun Jul 31 01:08:22 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:22.780620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-fa723551-af87-4e18-b882-a1323249fd6b tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e80c11940>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:23.018552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-fa723551-af87-4e18-b882-a1323249fd6b tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 692 microversion: 2.1 time: 0.264985 Jul 31 01:08:23.019285 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 11/22] 10.222.0.42 () {58 vars in 1170 bytes} [Sun Jul 31 01:08:22 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 692 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:23.046876 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2eb4b2dd-1c89-4376-ad77-ecb71072832d tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e803f39d0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:23.271960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2eb4b2dd-1c89-4376-ad77-ecb71072832d tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1032 microversion: 2.1 time: 0.235618 Jul 31 01:08:23.273709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Jul 31 01:08:23 2022] GET /compute/v2.1/os-volumes/detail => generated 1032 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:23.286660 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1db09fad-5c10-4799-94f0-afc485f08ffa tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e80c11ee0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:23.631402 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:23.632141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:23.648281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:23.649451 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:23.650677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:23.655764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:23.700130 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1db09fad-5c10-4799-94f0-afc485f08ffa tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.416280 Jul 31 01:08:23.700977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 12/24] 10.222.0.42 () {58 vars in 1188 bytes} [Sun Jul 31 01:08:23 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:23.727597 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-db7bfc43-cc45-4d42-a5e7-e5c2608f43c7 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] Calling method '.version_select at 0x7f3e80c204c0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:24.012106 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-db7bfc43-cc45-4d42-a5e7-e5c2608f43c7 tempest-VolumesTestJSON-264215241 tempest-VolumesTestJSON-264215241-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1032 microversion: 2.1 time: 0.296822 Jul 31 01:08:24.012686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Jul 31 01:08:23 2022] GET /compute/v2.1/os-volumes/detail => generated 1032 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:24.030273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2107271198", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b"}], "security_groups": [{"name": "tempest-securitygroup--1957011307"}], "key_name": "tempest-keypair-1298245109", "user_data": "}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:24.056788 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:24.080425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-800c423c-6eff-428a-824a-23901f382d14 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1267 microversion: 2.1 time: 0.453963 Jul 31 01:08:24.083560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 13/26] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:23 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1267 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:24.275085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] validate_networks() for [('3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:08:25.127307 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:25.133817 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:25.153172 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:25.161010 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:25.166079 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:25.166798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:25.351052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:25.392446 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cc65a53e-1345-4a02-b7d1-251ed6fab3e1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1267 microversion: 2.1 time: 0.272883 Jul 31 01:08:25.399511 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 14/27] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:25 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1267 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:25.869456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-2135532875", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "c3c043fa-9814-4058-ac46-ed026e54cf1b"}], "security_groups": [{"name": "tempest-securitygroup--486300412"}], "key_name": "tempest-keypair-460569984", "user_data": "}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:25.882974 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:08:25.883908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:08:25.883908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:08:25.884322 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:08:25.884744 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:08:25.900449 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:25.901159 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:25.902162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:25.902885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:25.941576 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:08:25.946591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:08:25.964552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:08:25.968632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:08:26.072303 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] validate_networks() for [('c3c043fa-9814-4058-ac46-ed026e54cf1b', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:08:26.074828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c2d88740-74ee-4d9b-9a38-12c7c281209a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.060977 Jul 31 01:08:26.078441 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 14/28] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:08:24 2022] POST /compute/v2.1/servers => generated 402 bytes in 2065 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:08:26.093642 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9f286bc3-939c-4c43-9786-d9e993ab9e3d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:26.094615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9f286bc3-939c-4c43-9786-d9e993ab9e3d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:26.266954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9f286bc3-939c-4c43-9786-d9e993ab9e3d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:26.279324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9f286bc3-939c-4c43-9786-d9e993ab9e3d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1242 microversion: 2.1 time: 0.191634 Jul 31 01:08:26.281536 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 15/29] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:26 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1242 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:26.449665 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:26.451015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:26.466105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:26.468718 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:26.469923 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:26.472086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:26.685265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:26.699386 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7acdc9fb-f000-4924-bc60-02348facc3c9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1267 microversion: 2.1 time: 0.258009 Jul 31 01:08:26.701001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 16/30] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:26 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1267 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:26.891264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:08:26.892176 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:08:26.893977 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:08:26.894795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:08:26.896367 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:08:26.925610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:26.931381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:26.958328 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:26.962495 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:27.037399 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for project 80ea98b340d94181b2193f4a79905988. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:08:27.043254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for user 56732ebe8ee24f3f9ce8f87746e8b161 and project 80ea98b340d94181b2193f4a79905988. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:08:27.061985 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:08:27.066149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:08:27.185230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f920cfe6-1bf8-4617-a451-71226368829f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.325521 Jul 31 01:08:27.196031 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 15/31] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Jul 31 01:08:25 2022] POST /compute/v2.1/servers => generated 401 bytes in 1337 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 31 01:08:27.219297 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-65fc5ed0-05a6-41a7-bf5c-ecb7d6fd35f4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:27.224626 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-65fc5ed0-05a6-41a7-bf5c-ecb7d6fd35f4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:27.323709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dce8b094-cce0-4af8-94a9-82003769cf93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:27.324768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dce8b094-cce0-4af8-94a9-82003769cf93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:27.443766 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-65fc5ed0-05a6-41a7-bf5c-ecb7d6fd35f4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:27.455516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-65fc5ed0-05a6-41a7-bf5c-ecb7d6fd35f4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:27.457440 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-65fc5ed0-05a6-41a7-bf5c-ecb7d6fd35f4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:27.458688 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-65fc5ed0-05a6-41a7-bf5c-ecb7d6fd35f4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1281 microversion: 2.20 time: 0.250227 Jul 31 01:08:27.477547 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 17/32] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:27 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1281 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:27.486886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dce8b094-cce0-4af8-94a9-82003769cf93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:27.497622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dce8b094-cce0-4af8-94a9-82003769cf93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1242 microversion: 2.1 time: 0.179716 Jul 31 01:08:27.498251 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 16/33] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:27 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1242 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:27.742149 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:27.750702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:27.818429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:27.819967 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:27.821327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:27.821979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:27.959736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:27.970113 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-388e36e3-6901-4685-9501-71dc31a0afcf tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1273 microversion: 2.1 time: 0.237384 Jul 31 01:08:27.998374 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 18/34] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:27 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1273 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:28.559985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:28.561630 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:28.570095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:28.571487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:28.578170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:28.585147 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:28.593307 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:28.595794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:28.631196 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:28.632172 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:28.639550 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:28.640407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:28.839780 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:28.899763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1e1db17a-dd8c-422b-b98f-f581f4652ef7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1246 microversion: 2.1 time: 0.348412 Jul 31 01:08:28.902367 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 17/35] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:28 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1246 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:28.922285 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:28.956772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:28.962596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:28.964710 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3e136572-b7b3-4ef0-8dce-a3d29bbf7726 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.409605 Jul 31 01:08:28.973884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 19/36] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:28 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:28.999295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:29.000290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:29.031302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:29.032152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:29.041972 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:29.042977 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:29.441249 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:29.454553 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-db824e72-8c59-4882-a053-c5f328975bdd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1273 microversion: 2.1 time: 0.461533 Jul 31 01:08:29.459730 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 18/37] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:28 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1273 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:29.946287 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:29.947435 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:29.958203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:29.960243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:29.968866 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:29.973432 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:30.023132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:30.023785 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:30.053382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:30.055938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:30.058326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:30.064276 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:30.250682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:30.270547 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7396eec9-8bec-4332-b01b-cef00d55027d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.331898 Jul 31 01:08:30.271151 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 20/38] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:29 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:30.295888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:30.318798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:30.320838 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:30.322032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8f57deb5-3736-4c0c-b639-cc67046af5ae tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.303167 Jul 31 01:08:30.322612 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 19/39] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:30 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:30.484875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:30.485767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:30.492787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:30.493323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:30.494365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:30.494867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:30.838737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:30.857311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-fa48b145-e190-4999-aef5-9f59757b293d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.376158 Jul 31 01:08:30.858937 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 21/40] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:30 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:31.293994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:31.295977 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:31.306755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:31.307536 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:31.308841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:31.309839 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:31.349742 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:31.351218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:31.363231 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:31.364010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:31.365343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:31.366432 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:31.412398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:31.422622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2a0726c7-cd0f-416e-a917-7f26395014d7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.132312 Jul 31 01:08:31.424048 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 20/41] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:31 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:31.493241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:31.509037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:31.512201 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:31.514422 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ffdaef35-5d90-400f-ab70-24374a59cf29 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.172957 Jul 31 01:08:31.515424 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 22/42] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:31 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:31.887944 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:31.888563 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:31.895261 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:31.895824 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:31.896655 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:31.897098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:32.344960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:32.367322 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-85e757f3-e022-420f-b89c-de3ca50294fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.483021 Jul 31 01:08:32.368346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 21/43] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:31 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:32.460091 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:32.462410 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:32.474209 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:32.475551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:32.477521 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:32.478676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:32.543021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:32.544416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:32.565340 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:32.566826 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:32.567927 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:32.568391 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:32.620082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:32.629781 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c719cd1e-f8ee-4684-b5bd-929c064a56da tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.177144 Jul 31 01:08:32.630988 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 23/44] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:32 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:32.727891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:32.746180 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:32.750029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:32.752692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-49ed9467-480b-4385-b843-d7dd37648603 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.214035 Jul 31 01:08:32.754245 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 22/45] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:32 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:33.427722 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:33.427722 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:33.429228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:33.429893 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:33.431332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:33.432205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:33.655525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:33.656572 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:33.657614 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:33.670115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:33.671891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:33.674865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-67f499e2-b536-49be-84f3-138bc15baace tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.268009 Jul 31 01:08:33.676086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 24/46] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:33 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:33.701109 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:33.702994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.027s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:33.793446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:33.794083 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:33.803272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:33.808376 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:33.810475 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:33.811610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:33.812224 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:33.819638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e123625a-94e4-4120-b732-5a36677ff1bb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.168171 Jul 31 01:08:33.822475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 23/47] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:33 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:33.934600 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:33.961489 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:33.973330 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:33.981238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-37864c07-5fc8-45e9-839b-0d11390c7e16 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.188554 Jul 31 01:08:33.984135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 25/48] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:33 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:34.722992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:34.724461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:34.733070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:34.733811 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:34.734741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:34.735295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:34.841788 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:34.842723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:34.851434 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:34.851904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:34.852723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:34.853172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:34.936633 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:34.949042 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c12c9e2e-efcd-4f34-87da-5b8722670777 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.111092 Jul 31 01:08:34.949644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 26/49] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:34 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:34.954541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:34.976090 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-84df5f10-780f-47e3-af15-5095af13795d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.258504 Jul 31 01:08:34.977513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 24/50] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:34 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:35.032377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:35.034356 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:35.047475 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:35.048929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:35.050584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:35.051271 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:35.212457 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:35.225122 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:35.230364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:35.231646 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-47dfd3b4-ed56-47a0-aa46-b92b347a71ef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.206059 Jul 31 01:08:35.232182 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 27/51] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:35 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:35.977832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:35.978804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:35.989135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:35.989873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:35.991143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:35.991998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:36.004143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:36.005274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:36.007758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:36.016796 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:36.018322 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:36.019814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:36.020652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:36.137484 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:36.154546 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6eb3b824-2069-4dd1-b426-800b12cc7e05 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.182865 Jul 31 01:08:36.159623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 25/52] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:35 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:36.257440 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:36.258915 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:36.293406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:36.297581 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:36.303795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:36.306363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:36.320441 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:36.333636 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7f6c5c11-adb8-4177-b984-380783bc9898 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.335595 Jul 31 01:08:36.354747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 28/53] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:35 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:36.458586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:36.488534 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:36.492447 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:36.495065 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-aca7edf7-bf63-4553-8e88-75508a36b0af tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.241261 Jul 31 01:08:36.496526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 26/54] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:36 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:37.179837 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:37.181003 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:37.190222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:37.190691 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:37.191614 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:37.192057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:37.259701 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:37.268627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-50ab524d-50c3-4f71-91b3-bdc2bea4b690 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.094601 Jul 31 01:08:37.269303 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 29/55] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:37 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:37.360308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:37.360958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:37.363753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=4 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:37.370197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:37.370649 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:37.371478 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:37.371923 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:37.497852 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=20 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:37.542246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:37.543431 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:37.544404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:37.559833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=16 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:37.565423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:37.566039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:37.567116 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:37.567719 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:37.569357 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4fc9a75d-cf5d-4c31-9d9f-5441236e82cd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.213010 Jul 31 01:08:37.570268 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 27/56] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:37 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:37.665249 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:37.678104 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:37.679705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:37.680871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b3e0f260-d2b1-4411-ac32-bdca98384f58 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.148524 Jul 31 01:08:37.681450 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:37 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:38.290035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:38.290622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:38.297202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:38.297953 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:38.298830 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:38.299452 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:38.418744 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:38.431280 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3324f1e3-5c11-448a-93c0-0776d98c752c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.144910 Jul 31 01:08:38.432108 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 28/58] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:38 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:38.608093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:38.609224 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:38.624037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:38.624791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:38.625878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:38.626722 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:38.708163 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:38.708792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:38.715392 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:38.715846 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:38.716619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:38.717055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:38.834171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:38.852601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:38.860256 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:38.870911 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c6a2ce17-0bc3-411a-a4fb-3ff6546319fd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.160280 Jul 31 01:08:38.871850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 29/59] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:38 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:38.900021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:38.912371 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d464a85e-48ff-4494-8b36-a711b94d5e5f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.315534 Jul 31 01:08:38.919945 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 31/60] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:38 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:39.462409 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:39.463101 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:39.469992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:39.470725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:39.471556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:39.471999 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:39.537148 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:39.541240 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:39.547505 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-97914c8b-f51e-4fd1-aa1f-bf4964ce10c8 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.088944 Jul 31 01:08:39.549250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 30/61] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:39 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:39.894544 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:39.895298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:39.902426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:39.902884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:39.903732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:39.904152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:39.909358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:39.962337 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:39.964334 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:39.985901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:39.987421 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:39.994645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:39.995559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:40.044809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:40.060668 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:40.063130 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:40.064693 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4d2446d8-5b27-426e-9e91-cd81f701e9a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.174203 Jul 31 01:08:40.065664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 32/62] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:39 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:40.315137 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:40.324373 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-967febda-4f7d-4cda-aa63-b4b4ef0a3b74 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.390841 Jul 31 01:08:40.325685 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 31/63] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:39 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:40.570891 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:40.571565 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:40.578502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:40.578995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:40.580355 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:40.580811 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:40.650797 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:40.664414 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-aa0944ff-e15c-4b4b-ad76-de21a23fa781 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.097005 Jul 31 01:08:40.665495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 33/64] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:40 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:41.103986 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:41.105320 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:41.112743 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:41.113390 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:41.114352 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:41.114912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:41.233085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:41.249492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:41.251087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:41.253038 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3a021e91-0bc0-4fb5-9e6f-d3224b6e2913 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.153714 Jul 31 01:08:41.253820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 32/65] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:41 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:41.359594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:41.360231 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:41.367080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:41.367622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:41.368405 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:41.368841 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:41.535653 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:41.544342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b2247a4c-ff5a-4bb3-ae28-371eae5fa19f tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.188948 Jul 31 01:08:41.544853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 34/66] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:41 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:41.699860 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:41.702112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:41.711801 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:41.712670 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:41.714416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:41.715115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:41.821804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:41.830310 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4a8b0138-1520-40d0-bbbf-5bc61fecd2cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.138652 Jul 31 01:08:41.830905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 33/67] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:41 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:42.285502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:42.286544 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:42.305465 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:42.312477 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:42.315827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:42.320747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:42.516358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:42.535571 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:42.563904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:42.563904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-62bc6c7a-6b59-489b-9e8e-53d6df209cec tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1285 microversion: 2.20 time: 0.275141 Jul 31 01:08:42.567993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 35/68] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:42 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1285 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:42.587560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:42.588414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:42.606198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:42.607115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:42.609607 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:42.610308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:42.838558 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:42.856252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:42.857750 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:42.871653 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:42.874118 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-45598b4b-a982-4be1-8f71-bd8d9e48f95d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.288307 Jul 31 01:08:42.874748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:42.876583 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 34/69] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:42 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:42.877789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:42.879280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:43.044265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:43.059758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f9f49ff1-4c2f-4ac0-9c82-87490d203e30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1238 microversion: 2.1 time: 0.209916 Jul 31 01:08:43.061407 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 36/70] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:42 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1238 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:43.604424 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:43.604424 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:43.613760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=7 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:43.621697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:43.623143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:43.625087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:43.625832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:43.833914 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:43.851054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:43.858773 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:43.863115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-00b9b2de-1f76-4150-9842-258dc5759636 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1277 microversion: 2.20 time: 0.269784 Jul 31 01:08:43.866720 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 35/71] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:43 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1277 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:43.926523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:43.928523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:43.961632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:43.963649 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:43.968905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:43.971587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:44.101129 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:44.102076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:44.119479 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:44.119871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:44.124016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:44.125313 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:44.295651 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:44.307224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fcdb90e1-02dd-4139-aae3-e449d6970edd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1294 microversion: 2.1 time: 0.212946 Jul 31 01:08:44.309708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 36/72] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:44 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1294 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:44.466748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:44.480744 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3cdc47ff-6669-4f19-a91c-76af00a85c71 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.571378 Jul 31 01:08:44.482410 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 37/73] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:43 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:44.906970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:44.908650 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:44.920933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:44.922248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:44.924629 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:44.925765 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:45.077121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:45.117174 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:45.123336 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:45.125708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f2c42009-cb30-4a26-b65b-04b137923f22 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1333 microversion: 2.20 time: 0.225940 Jul 31 01:08:45.134248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 37/74] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:44 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1333 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:45.355422 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:45.357643 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:45.373097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:45.375274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:45.377383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:45.378494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:45.516690 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:45.518776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:45.539395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:45.540689 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:45.542526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:45.557897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:45.576329 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:45.602050 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4b1e6286-8eaa-4b98-b41a-ae1dd3b84dc0 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1294 microversion: 2.1 time: 0.253552 Jul 31 01:08:45.615416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 38/75] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:45 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1294 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:45.936885 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:45.956831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-84d48d99-3028-4862-a552-e3832985680b tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.451484 Jul 31 01:08:45.958329 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 38/76] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:45 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:46.034692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-6d1e0940-d13b-4ada-a96e-76d17f75a4ea req-290314b3-956b-429b-9da1-ee4258659ac7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d86c18ba-e366-4ccb-823b-d3c453e49d6c", "tag": "5b3a5ff4-3c17-421a-8b45-bed2a774a705"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:46.055624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-6d1e0940-d13b-4ada-a96e-76d17f75a4ea req-290314b3-956b-429b-9da1-ee4258659ac7 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:46.056107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-6d1e0940-d13b-4ada-a96e-76d17f75a4ea req-290314b3-956b-429b-9da1-ee4258659ac7 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:46.087942 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-6d1e0940-d13b-4ada-a96e-76d17f75a4ea req-290314b3-956b-429b-9da1-ee4258659ac7 service nova] Creating event network-changed:5b3a5ff4-3c17-421a-8b45-bed2a774a705 for instance d86c18ba-e366-4ccb-823b-d3c453e49d6c on hv-cinder-maste Jul 31 01:08:46.089007 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-6d1e0940-d13b-4ada-a96e-76d17f75a4ea req-290314b3-956b-429b-9da1-ee4258659ac7 service nova] Acquired lock "compute-rpcapi-router" {{(pid=117943) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 31 01:08:46.091208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac0bd22-35fb-40af-afad-ab9e0493be88 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:46.092176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac0bd22-35fb-40af-afad-ab9e0493be88 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:46.093669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac0bd22-35fb-40af-afad-ab9e0493be88 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:46.095605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac0bd22-35fb-40af-afad-ab9e0493be88 None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:46.112965 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.compute.rpcapi [None req-3ac0bd22-35fb-40af-afad-ab9e0493be88 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 31 01:08:46.114542 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac0bd22-35fb-40af-afad-ab9e0493be88 None None] Releasing lock "compute-rpcapi-router" {{(pid=117943) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 31 01:08:46.145296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3ac0bd22-35fb-40af-afad-ab9e0493be88 None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.539178 Jul 31 01:08:46.147291 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 39/77] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:08:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:46.154484 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:46.155658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:46.167137 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:46.167770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:46.168582 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:46.169339 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:46.293380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:46.303198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:46.305388 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:46.306747 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e7e3479a-00e4-4790-9bfd-ea13e3df5cf4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1333 microversion: 2.20 time: 0.160166 Jul 31 01:08:46.307530 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 39/78] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:46 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1333 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:46.646581 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:46.647892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:46.660472 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:46.661605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:46.663494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:46.664604 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:46.772888 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:46.790512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-39468124-a05e-476d-814c-50b247b69b6b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1294 microversion: 2.1 time: 0.155038 Jul 31 01:08:46.791608 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 40/79] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:46 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1294 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:46.996634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:46.997497 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:47.003832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:47.004399 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:47.005139 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:47.005897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:47.222329 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:47.237460 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-06c6116f-f401-4bb7-acfa-55aeb2a4d6ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.244596 Jul 31 01:08:47.238788 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 40/80] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:46 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:47.350190 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:47.351495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:47.362609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:47.363680 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:47.365005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:47.365832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:47.475788 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:47.487673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:47.489512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:47.490818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bb2317e6-5278-498a-b939-475c70dafc9d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1333 microversion: 2.20 time: 0.148490 Jul 31 01:08:47.491843 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 41/81] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:47 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1333 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:47.830198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:47.831281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:47.841735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:47.842213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:47.843283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:47.843793 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:47.984045 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:47.993334 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4da81868-efdc-424a-92fe-b57fb2de380a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1294 microversion: 2.1 time: 0.170732 Jul 31 01:08:47.994532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 41/82] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:47 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1294 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:48.270530 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:48.271305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:48.273891 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=4 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:48.287702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:48.288145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:48.289168 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:48.289747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:48.472555 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:48.480212 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-34e95c91-97ef-447f-86c1-da907192f244 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.214040 Jul 31 01:08:48.486976 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 42/83] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:48 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:48.513909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:48.514531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:48.522302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:48.522749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:48.523573 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:48.524008 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:48.594349 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:48.610304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:48.613907 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:48.616613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f1811767-7679-4ce0-8dc0-fe6b922953a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1333 microversion: 2.20 time: 0.106414 Jul 31 01:08:48.617503 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 42/84] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:48 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1333 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:49.066702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:49.067929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:49.101383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:49.105317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:49.106811 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:49.107632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:49.274433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:49.290971 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-824b0f8d-d780-46b7-b04a-3ad1bd0fe792 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1294 microversion: 2.1 time: 0.233542 Jul 31 01:08:49.292665 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 43/85] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:49 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1294 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:49.509149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:49.510533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:49.523381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:49.525407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:49.527291 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:49.528362 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:49.659574 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:49.660856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:49.678596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:49.678596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:49.678596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:49.678596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:49.764574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:49.787409 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0cad3b21-1693-4798-b89b-46c4b84c96ae tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.286552 Jul 31 01:08:49.804369 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 43/86] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:49 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:49.868865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:49.884014 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:49.887138 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:49.889372 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-fe3c8451-7201-4b46-b62b-5c1655b0f818 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1333 microversion: 2.20 time: 0.244147 Jul 31 01:08:49.890419 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 44/87] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:49 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1333 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:08:50.327650 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:50.328704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:50.337835 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:50.344805 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:50.346982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:50.347679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:50.816309 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:50.828582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:50.830124 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:50.855931 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:50.856924 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:50.858382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-16d53de7-7a37-416e-a4e2-8e2abfcf442b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1312 microversion: 2.1 time: 0.533443 Jul 31 01:08:50.859183 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:50.860017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:50.861869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 44/88] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:50 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1312 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:50.936132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:50.939221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:50.957435 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:50.958945 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:50.961608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:50.963190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:51.216654 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:51.221077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:51.246098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:51.247409 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4a917e5c-13c2-4171-9006-2bce5db3689e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.425002 Jul 31 01:08:51.248950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 45/89] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:50 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:51.251899 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:51.254538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e1a25948-e397-46f4-b4c2-7785e3149493 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1339 microversion: 2.20 time: 0.342506 Jul 31 01:08:51.256051 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 45/90] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:50 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1339 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:51.912606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:51.913406 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:51.932241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:51.936880 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:51.938249 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:51.939439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:52.277627 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:52.279455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:52.309928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:52.314753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:52.316455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:52.317238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:52.443395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:52.464714 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e7625b0b-e1c3-4629-8ad6-2693418dfe59 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.562641 Jul 31 01:08:52.467651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 46/91] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:51 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:52.481602 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:52.483688 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:52.504304 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:52.505115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:52.506878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:52.509096 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:52.860582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:52.874158 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:52.876162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:52.877323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-500977cc-8897-4cc7-80cf-05754fd284ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1339 microversion: 2.20 time: 0.408182 Jul 31 01:08:52.878457 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 47/92] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:52 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1339 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:53.159740 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:53.188228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4c8f14cb-14a2-4811-b98b-e94504e27b7e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.921062 Jul 31 01:08:53.207783 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 46/93] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:52 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 929 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:53.307329 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-a8246c72-e069-4f65-955b-6e5ba82d5b1c req-224e3dba-f091-471b-a7d1-93f07919404f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ffec4de-cf74-4e4f-acc9-ea172ae3e171", "tag": "e754ffa9-197f-4f94-a875-08f75b6e8593"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:53.331895 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a8246c72-e069-4f65-955b-6e5ba82d5b1c req-224e3dba-f091-471b-a7d1-93f07919404f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:53.332400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a8246c72-e069-4f65-955b-6e5ba82d5b1c req-224e3dba-f091-471b-a7d1-93f07919404f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:53.395453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-a8246c72-e069-4f65-955b-6e5ba82d5b1c req-224e3dba-f091-471b-a7d1-93f07919404f service nova] Creating event network-changed:e754ffa9-197f-4f94-a875-08f75b6e8593 for instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 on hv-cinder-maste Jul 31 01:08:53.406830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-a8246c72-e069-4f65-955b-6e5ba82d5b1c req-224e3dba-f091-471b-a7d1-93f07919404f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.104439 Jul 31 01:08:53.408627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 48/94] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:08:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:53.507006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:53.514380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:53.534373 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:53.535807 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:53.537667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:53.538946 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:53.587556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:53.817785 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:53.835283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fb096478-b361-4624-929f-e1249db6eb7e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.336175 Jul 31 01:08:53.836972 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 47/95] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:53 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:53.902428 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:53.903641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:53.911726 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:53.912434 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:53.913525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:53.914032 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:54.037875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:54.042505 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:54.050945 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:54.054429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:54.055977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-000b2b39-b4b2-464b-8ae2-182bcfeb9158 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1339 microversion: 2.20 time: 0.158240 Jul 31 01:08:54.056709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 49/96] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:53 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1339 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:54.221695 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:54.223955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:54.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:54.237264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:54.238116 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:54.238564 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:54.452706 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:54.460217 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4ec58477-8280-4acc-a263-8edb98cec2a2 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.242866 Jul 31 01:08:54.460964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 48/97] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:54 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:54.868787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:54.869492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:54.877224 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:54.877953 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:54.878976 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:54.879517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:55.083050 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:55.085449 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:55.095836 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:55.097087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:55.097860 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:55.099408 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:55.100386 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:55.124298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c6248c47-9991-49a1-a0eb-5e319a4bc63e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.262649 Jul 31 01:08:55.136186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 50/98] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:54 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:55.345318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:55.357149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:55.359069 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:55.362236 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0bb9b8d2-e462-4f58-974b-9506791c1b51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.287011 Jul 31 01:08:55.375421 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 49/99] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:55 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:55.487690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:55.489233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:55.501286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:55.502628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:55.504410 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:55.505439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:55.766645 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:55.779252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2d335b69-c087-4bc5-930d-e68297a2a8de tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.299565 Jul 31 01:08:55.783427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 51/100] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:55 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:56.154395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:56.155091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:56.168623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:56.169231 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:56.170086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:56.170537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:56.338775 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:56.347538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9271814d-234d-4ef4-a5ba-9e0bddeef047 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.198477 Jul 31 01:08:56.348193 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 50/101] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:56 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:08:56.389677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:56.390324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:56.393032 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:08:56.399511 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:56.400501 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:56.401611 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:56.402370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:56.654914 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:56.672132 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:56.674682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:56.676285 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4a5d9276-d1a5-45ba-b73a-3e68de58c1aa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.291953 Jul 31 01:08:56.677460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 52/102] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:56 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:56.823631 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:56.827800 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:56.865986 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:56.866457 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:56.867318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:56.867844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:57.384796 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:57.389069 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:57.397978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:57.399888 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:57.401006 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:57.401746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:57.700583 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:57.711216 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c839e8eb-d4de-47cd-8c64-997b5a83683f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.330702 Jul 31 01:08:57.713927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 53/103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:57 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:57.722617 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:57.723856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:57.755220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:57.756098 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:57.757579 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:57.758205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:57.863959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:57.875450 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3f47eb5d-bf39-40e0-921d-b598f1d87829 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 1.060767 Jul 31 01:08:57.879855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 51/104] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:56 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 1066 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:58.122819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:58.146471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:58.150070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:58.153734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3622599a-5d93-4d95-ae96-d0457a138610 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.437732 Jul 31 01:08:58.155207 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 54/105] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:57 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:08:58.770966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:58.775496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:58.797503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:58.799331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:58.801244 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:58.802685 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:58.839272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-157af937-a102-49b8-9496-999c90a7d3b2 req-d6864161-208b-4f50-9719-17ee5c320e0e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ef19c2df-372c-4c77-ad6c-ada1cd419a14", "tag": "f5779c70-0ae8-4a58-a8ba-25f9117da52e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:08:58.865126 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-157af937-a102-49b8-9496-999c90a7d3b2 req-d6864161-208b-4f50-9719-17ee5c320e0e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:58.866866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-157af937-a102-49b8-9496-999c90a7d3b2 req-d6864161-208b-4f50-9719-17ee5c320e0e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:58.896070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-157af937-a102-49b8-9496-999c90a7d3b2 req-d6864161-208b-4f50-9719-17ee5c320e0e service nova] Creating event network-changed:f5779c70-0ae8-4a58-a8ba-25f9117da52e for instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 on hv-cinder-maste Jul 31 01:08:58.896904 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-157af937-a102-49b8-9496-999c90a7d3b2 req-d6864161-208b-4f50-9719-17ee5c320e0e service nova] Acquired lock "compute-rpcapi-router" {{(pid=117942) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 31 01:08:58.898323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d865cb79-ca0d-4475-a93d-f9ca702b0c0f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:58.899101 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d865cb79-ca0d-4475-a93d-f9ca702b0c0f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:58.900507 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d865cb79-ca0d-4475-a93d-f9ca702b0c0f None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:58.901143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d865cb79-ca0d-4475-a93d-f9ca702b0c0f None None] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:58.914777 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.compute.rpcapi [None req-d865cb79-ca0d-4475-a93d-f9ca702b0c0f None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 31 01:08:58.916053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d865cb79-ca0d-4475-a93d-f9ca702b0c0f None None] Releasing lock "compute-rpcapi-router" {{(pid=117942) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 31 01:08:58.962731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d865cb79-ca0d-4475-a93d-f9ca702b0c0f None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.236353 Jul 31 01:08:58.964049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 52/106] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:08:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:08:58.972645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:58.973590 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:58.996399 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:58.997147 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:58.998257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:58.998948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:59.037819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:59.070652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-30222d26-b0c5-45e5-ad5b-e201ebc51074 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.313409 Jul 31 01:08:59.071954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 55/107] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:58 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:59.182524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:08:59.183474 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:08:59.194590 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:59.195686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:59.196734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:08:59.197337 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:08:59.265437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:59.274038 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-221d1b19-0925-4141-b626-0088439a4849 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.308489 Jul 31 01:08:59.274728 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 53/108] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:08:58 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:08:59.405259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:59.414782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:59.416945 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:08:59.418582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8bacb7fd-3eba-444d-907f-246aeaf86d6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.241043 Jul 31 01:08:59.431915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 56/109] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:08:59 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:00.093569 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:00.094334 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:00.108303 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:00.108992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:00.111239 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:00.111820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:00.294792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:00.295622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:00.312957 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:00.313444 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:00.314255 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:00.319005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:00.348284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:00.363394 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-51560e42-48d6-4202-a935-6f913bff7a84 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.273731 Jul 31 01:09:00.373186 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 54/110] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:00 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:00.448602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:00.449176 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:00.461003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:00.461579 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:00.462651 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:00.463741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:00.703442 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:00.720828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-57b1ee74-deb5-4c82-a61e-89240ad0f73e tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.423941 Jul 31 01:09:00.728382 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 57/111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:00 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:01.069009 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:01.107201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:01.124429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:01.129260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5db247c3-2aef-41c9-9ea1-258f51bf6e5b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.689293 Jul 31 01:09:01.133375 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 55/112] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:00 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 696 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:01.398843 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:01.399920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:01.426806 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:01.429682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:01.432771 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:01.434978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:01.736591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:01.745540 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-23cbedc4-7877-47fc-b6e3-adc07052be99 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.354846 Jul 31 01:09:01.759708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 58/113] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:01 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:01.775015 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:01.776107 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:01.787134 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:01.787822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:01.789562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:01.790253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:02.055041 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:02.070470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fa7ead75-cafb-499e-859f-344786deae7d tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.305182 Jul 31 01:09:02.071717 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 56/114] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:01 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:02.179917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:02.180552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:02.200077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:02.201749 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:02.204037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:02.205538 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:02.556096 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:02.596293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:02.600108 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:02.605833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2277bf46-abed-41ce-8f6a-c05e0c064746 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.428965 Jul 31 01:09:02.607131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 59/115] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:02 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:02.776179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:02.778935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:02.830057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:02.830616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:02.834282 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:02.835245 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:03.108273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:03.109199 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:03.121268 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:03.123053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:03.123986 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:03.124432 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:03.185873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:03.207799 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f1bc7e7f-807f-4007-81f8-d940d1b52600 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.441482 Jul 31 01:09:03.209747 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 57/116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:02 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:03.409723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:03.419586 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0de05ebc-6186-48da-9db7-aebcfa3ca471 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.316044 Jul 31 01:09:03.420537 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 60/117] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:03 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:03.632700 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:03.633854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:03.642903 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:03.643755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:03.644843 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:03.645421 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:03.836012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:03.846293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:03.848723 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:03.849889 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0850af2e-cb6e-41ec-b1eb-ceff982d1f9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.222813 Jul 31 01:09:03.850460 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 58/118] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:03 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:04.246479 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:04.248122 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:04.261888 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:04.263522 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:04.265448 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:04.266077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:04.443117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:04.444381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:04.453054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:04.453660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:04.454801 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:04.455566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:04.629814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:04.644564 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c5428a64-9f19-4fe4-8c6e-b522289607af tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.406650 Jul 31 01:09:04.645857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 61/119] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:04 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:04.725509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:04.740531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cc1bcb62-fef0-4802-ba7a-2b59b8dc0157 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.302428 Jul 31 01:09:04.752506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 59/120] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:04 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:04.879668 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:04.882360 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:04.898714 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:04.899529 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:04.920257 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:04.921127 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:05.127570 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:05.138042 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:05.140342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:05.142595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-631ba4ff-4885-4699-91e2-0757320766dc tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.271413 Jul 31 01:09:05.143697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 62/121] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:04 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:05.673637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:05.674713 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:05.683835 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:05.684626 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:05.685696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:05.686273 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:05.764265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:05.765512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:05.774611 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:05.775223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:05.776106 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:05.776550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:05.899459 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:05.908960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7ad42d9e-d54e-4ec8-b3a1-0daa8ca34601 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.240189 Jul 31 01:09:05.909634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 60/122] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:05 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:05.992528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:06.005142 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c06eb5ae-5aca-4aa0-be07-a217ef14ffec tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.245549 Jul 31 01:09:06.009545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 63/123] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:05 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:06.168751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:06.170483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:06.183660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:06.185234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:06.187551 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:06.188748 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:06.421854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:06.431873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:06.433347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:06.434886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-bf95a804-48e8-44bf-833b-42a0c2118622 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.272381 Jul 31 01:09:06.435515 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 61/124] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:06 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:06.937941 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:06.947620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:06.970933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:06.975420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:06.977872 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:06.978584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:07.032776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:07.035108 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:07.044371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:07.045808 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:07.047290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:07.048683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:07.208057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:07.226934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d6b557cb-4508-443f-96ab-f99734920c06 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.299326 Jul 31 01:09:07.227888 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 64/125] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:06 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:07.305663 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:07.320250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f01666bd-1772-4f4b-a12e-f5c936246ef0 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.296027 Jul 31 01:09:07.329479 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 62/126] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:07 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:07.477539 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:07.478282 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:07.489114 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:07.489919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:07.491297 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:07.492099 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:07.696272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:07.710536 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:07.713089 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:07.714758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6df6d018-e481-4633-890b-b13e505b46cf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.244004 Jul 31 01:09:07.716167 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 65/127] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:07 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:08.265765 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:08.267465 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:08.297559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:08.298687 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:08.306156 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:08.310794 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:08.366316 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:08.378062 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:08.400493 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:08.401283 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:08.403977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:08.408286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:08.437666 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:08.695684 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:08.705005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7138ec5b-1c92-42bd-b2e0-c7124c6eed13 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.446355 Jul 31 01:09:08.706379 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 63/128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:08 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:08.760207 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:08.772626 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4feadbdc-3a2b-4a4d-8b6b-0b9043212e10 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.421396 Jul 31 01:09:08.773669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 66/129] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:08 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:08.779849 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:08.780791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:08.789528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:08.790280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:08.791315 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:08.792202 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:08.885267 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-5b1c08f6-90d7-4024-87a7-268c839206f2 req-00d92c7a-36df-4afe-ad55-008903ef8279 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d86c18ba-e366-4ccb-823b-d3c453e49d6c", "name": "network-vif-plugged", "status": "completed", "tag": "5b3a5ff4-3c17-421a-8b45-bed2a774a705"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:09:08.934857 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-5b1c08f6-90d7-4024-87a7-268c839206f2 req-00d92c7a-36df-4afe-ad55-008903ef8279 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:08.935940 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-5b1c08f6-90d7-4024-87a7-268c839206f2 req-00d92c7a-36df-4afe-ad55-008903ef8279 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:08.945961 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:08.969701 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-5b1c08f6-90d7-4024-87a7-268c839206f2 req-00d92c7a-36df-4afe-ad55-008903ef8279 service nova] Creating event network-vif-plugged:5b3a5ff4-3c17-421a-8b45-bed2a774a705 for instance d86c18ba-e366-4ccb-823b-d3c453e49d6c on hv-cinder-maste Jul 31 01:09:08.984630 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-5b1c08f6-90d7-4024-87a7-268c839206f2 req-00d92c7a-36df-4afe-ad55-008903ef8279 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.273973 Jul 31 01:09:08.987806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 64/130] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:09:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:09:09.013807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:09.024787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:09.026687 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:09.043913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f78bf906-d3f1-4290-9fc2-dec3b63d68de tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.268891 Jul 31 01:09:09.052396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 67/131] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:08 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:09.749694 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:09.754837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:09.784626 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:09.785586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:09.787229 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:09.790953 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:09.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:09.807447 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:09.808715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:09.840940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:09.841831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:09.843404 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:09.844266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:10.051048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:10.064402 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6772e980-d43e-4e5f-b2b0-93e41d47efc1 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.266755 Jul 31 01:09:10.065053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 68/132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:09 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:10.098086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:10.099022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:10.100383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:10.108491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-512a789f-772d-4a7b-82bc-ea1db3571202 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.369026 Jul 31 01:09:10.110790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:10.111510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:10.113385 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:10.114153 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:10.116006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 65/133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:09 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:10.295906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:10.310388 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:10.312575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:10.314603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-76255751-9f46-45a7-b665-d1d0f537d31a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.223356 Jul 31 01:09:10.315870 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 69/134] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:10 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:11.115404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:11.116593 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:11.143936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:11.145184 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:11.146919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:11.147935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:11.150595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:11.153530 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:11.182376 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:11.190113 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:11.196728 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:11.199316 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:11.466966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:11.485055 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:11.489933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e55247fe-a01f-4187-9699-85cc3ecd9a84 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.381635 Jul 31 01:09:11.494729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-beb112fa-cef6-49f6-8d6a-fc768dc9e795 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.366130 Jul 31 01:09:11.497240 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 70/135] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:11 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:11.503658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:11.504771 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:11.510640 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 66/136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:11 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:11.513771 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:11.514712 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:11.516091 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:11.517548 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:11.725897 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:11.740225 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:11.744268 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:11.747626 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4617e89d-1c93-4dcf-bee0-26a65cb362d8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.248857 Jul 31 01:09:11.748346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 71/137] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:11 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:12.205781 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-b43e067d-85b1-463c-84d0-0afa1e458395 req-f102f3f5-ca4e-466c-a1e4-b88bfaf37604 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ffec4de-cf74-4e4f-acc9-ea172ae3e171", "name": "network-vif-plugged", "status": "completed", "tag": "e754ffa9-197f-4f94-a875-08f75b6e8593"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:09:12.224306 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b43e067d-85b1-463c-84d0-0afa1e458395 req-f102f3f5-ca4e-466c-a1e4-b88bfaf37604 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:12.225145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b43e067d-85b1-463c-84d0-0afa1e458395 req-f102f3f5-ca4e-466c-a1e4-b88bfaf37604 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:12.243611 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-b43e067d-85b1-463c-84d0-0afa1e458395 req-f102f3f5-ca4e-466c-a1e4-b88bfaf37604 service nova] Creating event network-vif-plugged:e754ffa9-197f-4f94-a875-08f75b6e8593 for instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 on hv-cinder-maste Jul 31 01:09:12.253609 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-b43e067d-85b1-463c-84d0-0afa1e458395 req-f102f3f5-ca4e-466c-a1e4-b88bfaf37604 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054363 Jul 31 01:09:12.254628 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 67/138] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:09:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:09:12.541921 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:12.542470 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:12.544226 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:12.549625 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:12.562574 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:12.563883 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:12.565833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:12.567404 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:12.568647 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:12.569659 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:12.573151 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:12.574061 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:12.896674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:12.909529 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b3afce18-9ba6-4f88-9583-c8b3c5a5ba44 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 0.378901 Jul 31 01:09:12.920705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 72/139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:12 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:12.932172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:12.933236 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:12.942223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:12.943032 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:12.944265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:12.944930 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:13.037722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:13.046268 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9dd9b4af-78b3-4572-8cd4-9f83bf8bd258 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.516959 Jul 31 01:09:13.062929 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 68/140] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:12 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:13.382243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:13.413327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:13.418724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:13.438947 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4f323445-00d5-4821-9359-cf863240a633 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.515509 Jul 31 01:09:13.459408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 73/141] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:12 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:13.986091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:13.989254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:13.993693 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=68 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:14.002125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:14.002923 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:14.003994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:14.004630 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:14.073690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:14.075888 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:14.087220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:14.088044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:14.089471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:14.090253 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:14.349558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:14.364995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a051a209-8367-4482-ba4a-0f2c54065df9 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1315 microversion: 2.1 time: 0.298302 Jul 31 01:09:14.367050 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 74/142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:14 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1315 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:14.476187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:14.477318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:14.485431 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:14.486254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:14.487068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:14.487620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:14.761266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:14.801567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:14.811707 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:14.820668 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e6e9e94e-5a8f-45df-ad0c-11201f96d23c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.349431 Jul 31 01:09:14.822543 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 75/143] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:14 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:15.114247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:15.137438 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c3f2b521-5c46-47fa-aeb6-63c925b19191 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1368 microversion: 2.1 time: 1.160735 Jul 31 01:09:15.142823 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 69/144] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:13 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1368 bytes in 1167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:15.397214 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:15.399758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:15.414047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:15.415469 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:15.417160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:15.418174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:15.856325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:15.867239 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:15.869204 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:15.884852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1d711c39-d706-435a-97c8-6b4b891b23b5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1501 microversion: 2.1 time: 0.498524 Jul 31 01:09:15.889517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 76/145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:15 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1501 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:15.912387 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:15.916053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:15.916053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:15.922959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:16.164883 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:16.165503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:16.179562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:16.179955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:16.180506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:16.194983 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:16.572611 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:16.594781 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:16.624201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:16.631403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1e2135d5-dfcc-45c9-92ba-ad894d79a509 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.770469 Jul 31 01:09:16.633134 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 70/146] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:15 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:16.647818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-2887894f-b2e4-4153-96a5-781382b718a3 req-44db5d34-2993-47d6-9064-a30f8c8122f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ef19c2df-372c-4c77-ad6c-ada1cd419a14", "name": "network-vif-plugged", "status": "completed", "tag": "f5779c70-0ae8-4a58-a8ba-25f9117da52e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:09:16.675419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-2887894f-b2e4-4153-96a5-781382b718a3 req-44db5d34-2993-47d6-9064-a30f8c8122f1 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:16.677062 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-2887894f-b2e4-4153-96a5-781382b718a3 req-44db5d34-2993-47d6-9064-a30f8c8122f1 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:16.709924 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-2887894f-b2e4-4153-96a5-781382b718a3 req-44db5d34-2993-47d6-9064-a30f8c8122f1 service nova] Creating event network-vif-plugged:f5779c70-0ae8-4a58-a8ba-25f9117da52e for instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 on hv-cinder-maste Jul 31 01:09:16.724194 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-2887894f-b2e4-4153-96a5-781382b718a3 req-44db5d34-2993-47d6-9064-a30f8c8122f1 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087043 Jul 31 01:09:16.725593 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 71/147] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:09:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:09:16.940071 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:16.952599 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-eba96b4a-a458-4db3-a35a-bf9744f51452 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1559 microversion: 2.1 time: 0.792120 Jul 31 01:09:16.967389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 77/148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:16 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1559 bytes in 797 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:16.988255 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fc4a5e56-363b-47cb-b2a2-26a1e0f7a6ae tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:16.992881 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fc4a5e56-363b-47cb-b2a2-26a1e0f7a6ae tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:17.033143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e56-363b-47cb-b2a2-26a1e0f7a6ae tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:17.034412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e56-363b-47cb-b2a2-26a1e0f7a6ae tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:17.038230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e56-363b-47cb-b2a2-26a1e0f7a6ae tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:17.041228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e56-363b-47cb-b2a2-26a1e0f7a6ae tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:17.395753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fc4a5e56-363b-47cb-b2a2-26a1e0f7a6ae tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.418253 Jul 31 01:09:17.404264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 72/149] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:09:16 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-interface => generated 285 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:09:17.660508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:17.661652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:17.698991 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:17.701936 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:17.702768 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:17.704770 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:17.970652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:17.989821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:17.993005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:17.995139 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5d995c36-7d9a-44f6-9270-a8c3fa6a9e92 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.338457 Jul 31 01:09:17.996592 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 78/150] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:17 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:19.030297 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:19.032433 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:19.042565 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:19.043962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:19.045382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:19.046492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:19.844844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:19.875698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:19.883485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:19.891781 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a7c98307-75ac-4a20-991d-0dc6afadae1a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.867402 Jul 31 01:09:19.894943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 73/151] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:19 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 873 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:20.167638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-1cfee2c8-60d6-431b-9d6a-dcf37fc12745 req-5e345c03-f55f-4ce1-83d1-90096f50f239 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ffec4de-cf74-4e4f-acc9-ea172ae3e171", "tag": "e754ffa9-197f-4f94-a875-08f75b6e8593"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:09:20.234246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=74 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:20.246987 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-1cfee2c8-60d6-431b-9d6a-dcf37fc12745 req-5e345c03-f55f-4ce1-83d1-90096f50f239 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:20.248667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-1cfee2c8-60d6-431b-9d6a-dcf37fc12745 req-5e345c03-f55f-4ce1-83d1-90096f50f239 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:20.295339 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-1cfee2c8-60d6-431b-9d6a-dcf37fc12745 req-5e345c03-f55f-4ce1-83d1-90096f50f239 service nova] Creating event network-changed:e754ffa9-197f-4f94-a875-08f75b6e8593 for instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 on hv-cinder-maste Jul 31 01:09:20.316345 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-1cfee2c8-60d6-431b-9d6a-dcf37fc12745 req-5e345c03-f55f-4ce1-83d1-90096f50f239 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.160057 Jul 31 01:09:20.318252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 79/152] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:09:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:09:20.941697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:20.945328 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:20.954692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:20.955496 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:20.956845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:20.957524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:21.048608 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3bd20ca2-7718-4e99-8304-224fda5043a5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "09081d6c-6541-484d-801c-77b1a0145129"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:09:21.053581 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3bd20ca2-7718-4e99-8304-224fda5043a5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:21.065922 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3bd20ca2-7718-4e99-8304-224fda5043a5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:21.066682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3bd20ca2-7718-4e99-8304-224fda5043a5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:21.067787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3bd20ca2-7718-4e99-8304-224fda5043a5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:21.068652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3bd20ca2-7718-4e99-8304-224fda5043a5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:21.338102 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:21.353714 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:21.355219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:21.356663 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ad49e15c-b8b4-49f5-9b01-bb647a7c167a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.429190 Jul 31 01:09:21.369354 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 74/153] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:20 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:09:22.382908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:22.383925 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:22.394181 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:22.395293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:22.396673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:22.397417 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:22.732808 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:22.743633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:22.745760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:22.747932 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-04a6b747-c71e-42db-ac29-cf67a7dfecc2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.370872 Jul 31 01:09:22.748661 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 75/154] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:22 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:23.776669 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:23.777842 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:23.790689 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:23.791425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:23.792842 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:23.794000 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:23.969820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:23.980570 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:23.982086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:23.983271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-45ff3198-c95b-433e-acd3-63f5758a1195 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.215222 Jul 31 01:09:23.983870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 76/155] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:23 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:25.022575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:25.023228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:25.029711 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:25.030164 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:25.030954 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:25.031951 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:25.174039 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:25.182968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:25.184950 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:25.186096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dce37435-f8d1-498f-bf25-19cb324b0860 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.167470 Jul 31 01:09:25.186682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 77/156] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:25 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:26.203097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3bd20ca2-7718-4e99-8304-224fda5043a5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "POST /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 5.163457 Jul 31 01:09:26.204433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 80/157] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:09:21 2022] POST /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c/os-volume_attachments => generated 194 bytes in 5167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:09:26.211763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:26.213982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:26.223001 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:26.223440 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:26.224762 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:26.225348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:26.580352 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:26.589473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:26.591046 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:26.592256 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b8a45bdd-f883-405e-af8b-706612081497 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1406 microversion: 2.20 time: 0.385516 Jul 31 01:09:26.593175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 78/158] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:26 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1406 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:27.618609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:27.619343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:27.628417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:27.629024 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:27.630022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:27.630557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:27.820624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:27.829822 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:27.831530 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:27.835152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6c29517d-57d8-428c-aea9-00db43289907 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1598 microversion: 2.20 time: 0.221149 Jul 31 01:09:27.835152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 81/159] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:09:27 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1598 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:09:27.869088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-66b843b1-b7c3-4f77-a962-327471e1cb45 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:27.869861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-66b843b1-b7c3-4f77-a962-327471e1cb45 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:27.877226 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-66b843b1-b7c3-4f77-a962-327471e1cb45 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:27.877682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-66b843b1-b7c3-4f77-a962-327471e1cb45 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:27.878634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-66b843b1-b7c3-4f77-a962-327471e1cb45 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:27.879220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-66b843b1-b7c3-4f77-a962-327471e1cb45 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:27.967899 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-66b843b1-b7c3-4f77-a962-327471e1cb45 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.105004 Jul 31 01:09:27.968530 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 79/160] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Jul 31 01:09:27 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:09:29.392115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-4bd5c36d-1339-4849-bca5-77a7e2f84c59 req-d16bcc39-42c0-400f-bf6e-cabf415fe06e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ef19c2df-372c-4c77-ad6c-ada1cd419a14", "tag": "f5779c70-0ae8-4a58-a8ba-25f9117da52e"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:09:29.402439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-4bd5c36d-1339-4849-bca5-77a7e2f84c59 req-d16bcc39-42c0-400f-bf6e-cabf415fe06e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:29.402892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-4bd5c36d-1339-4849-bca5-77a7e2f84c59 req-d16bcc39-42c0-400f-bf6e-cabf415fe06e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:29.416281 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-4bd5c36d-1339-4849-bca5-77a7e2f84c59 req-d16bcc39-42c0-400f-bf6e-cabf415fe06e service nova] Creating event network-changed:f5779c70-0ae8-4a58-a8ba-25f9117da52e for instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 on hv-cinder-maste Jul 31 01:09:29.422743 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-4bd5c36d-1339-4849-bca5-77a7e2f84c59 req-d16bcc39-42c0-400f-bf6e-cabf415fe06e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033622 Jul 31 01:09:29.423561 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 82/161] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:09:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:09:45.501907 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:45.502820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:45.505941 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=21 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:45.507985 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=76 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:45.520562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:45.522379 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:45.523951 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:45.525163 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:45.572827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:09:45.725464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b511510c-7ccb-4cac-a71b-37e9a80a675c tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 204 len: 0 microversion: 2.1 time: 0.230191 Jul 31 01:09:45.726470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 80/162] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:09:45 2022] DELETE /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:09:45.736521 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:45.737151 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:45.741202 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=7 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:45.742227 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:09:45.750852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:45.751867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:45.752868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:45.753816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:45.942824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:45.953766 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b93ed74e-9fe1-46bd-84da-13de2d715e97 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1553 microversion: 2.1 time: 0.222091 Jul 31 01:09:45.954662 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 83/163] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:45 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1553 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:46.979283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:46.980476 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:46.988957 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:46.989532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:46.990434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:46.990989 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:47.460332 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:47.475656 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5dfd440c-725c-48ff-8d32-48d4b82641be tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1553 microversion: 2.1 time: 0.501913 Jul 31 01:09:47.476845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 81/164] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:46 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1553 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:09:48.519987 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:48.522043 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:48.537701 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:48.538439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:48.539644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:48.540392 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:48.742002 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:48.756323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f042b659-20e4-4cfa-a860-2836062bfb02 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1553 microversion: 2.1 time: 0.243451 Jul 31 01:09:48.757401 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 84/165] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:48 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1553 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:49.784391 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:49.786526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:49.797309 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:49.798091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:49.799906 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:49.800868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:50.083316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:50.102163 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0feca65e-bdca-40c0-bec3-cac74ede02d5 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1553 microversion: 2.1 time: 0.324688 Jul 31 01:09:50.103957 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 82/166] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:49 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1553 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:51.131396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:51.132883 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:51.142847 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:51.143624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:51.144602 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:51.145241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:51.473734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:51.497352 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-efc8dcb6-6d23-4298-bdd8-de9fe7820323 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1553 microversion: 2.1 time: 0.371168 Jul 31 01:09:51.515389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 85/167] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:51 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1553 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:52.509041 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-12df440c-e7e4-4676-aad8-d02080466a95 req-78a073e6-ec74-4699-b46f-48ad866e1219 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d86c18ba-e366-4ccb-823b-d3c453e49d6c", "name": "network-vif-deleted", "tag": "5b3a5ff4-3c17-421a-8b45-bed2a774a705"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:09:52.528829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-12df440c-e7e4-4676-aad8-d02080466a95 req-78a073e6-ec74-4699-b46f-48ad866e1219 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:52.529660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-12df440c-e7e4-4676-aad8-d02080466a95 req-78a073e6-ec74-4699-b46f-48ad866e1219 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:52.544259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:52.547567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:52.562549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:52.563859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:52.566031 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:52.567105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:52.617473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-12df440c-e7e4-4676-aad8-d02080466a95 req-78a073e6-ec74-4699-b46f-48ad866e1219 service nova] Creating event network-vif-deleted:5b3a5ff4-3c17-421a-8b45-bed2a774a705 for instance d86c18ba-e366-4ccb-823b-d3c453e49d6c on hv-cinder-maste Jul 31 01:09:52.629691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-12df440c-e7e4-4676-aad8-d02080466a95 req-78a073e6-ec74-4699-b46f-48ad866e1219 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.136454 Jul 31 01:09:52.631369 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 83/168] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:09:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:09:52.741997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:52.759780 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-823bd7c5-c276-402c-8f4a-08be0ba92172 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1346 microversion: 2.1 time: 0.231373 Jul 31 01:09:52.760669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 86/169] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:52 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1346 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:53.800463 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:53.801401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:53.823289 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:53.823779 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:53.824590 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:53.825033 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:53.880105 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:53.880571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:53.881043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lazy-loading 'fault' on Instance uuid d86c18ba-e366-4ccb-823b-d3c453e49d6c {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:09:53.985793 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4a002047e3e42338e4f57ba49e877fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '66c7f61afd0d4650bd530eff18a2318b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:09:53.999912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-17fb438b-4888-4a65-8acd-b7acbad856a8 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 200 len: 1351 microversion: 2.1 time: 0.209064 Jul 31 01:09:54.000528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 84/170] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:53 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 1351 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:09:55.036429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:55.038203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:55.049001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:55.050442 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:55.051728 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:55.052680 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:55.095880 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] HTTP exception thrown: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Request method failure captured: Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c HTTP/1.1 Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dX7IQPomRQzRAIfEJmiewWpdR29NMFQSclNoB4VWMYfaacPRuQGX-SE70k1cvEKHjfRUxq60oQj1emNQ8BJG6xfVPQbxWhbD2MYu6EOXEoi3QjHkjIA7qOdIUo4H0vl8jW70Ga3cJIa-y7Hs7LTQSdFXRu-znCdKjSuaiE8HP9aUdw Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:09:55.119005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_66c7f61afd0d4650bd530eff18a2318b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: c4a002047e3e42338e4f57ba49e877fb Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.120503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'd86c18ba-e366-4ccb-823b-d3c453e49d6c'} Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:09:55.121187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.122459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.123615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Returning 404 to user: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:09:55.123615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9ba767a6-85b5-43f8-a835-3cbe812939ca tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 404 len: 111 microversion: 2.1 time: 0.091587 Jul 31 01:09:55.125671 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 87/171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:55 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:09:55.442301 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:55.442301 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:55.449003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:55.449442 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:55.451543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:55.452623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:55.480341 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] HTTP exception thrown: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Request method failure captured: Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c/os-volume_attachments HTTP/1.1 Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dX7IQPomRQzRAIfEJmiewWpdR29NMFQSclNoB4VWMYfaacPRuQGX-SE70k1cvEKHjfRUxq60oQj1emNQ8BJG6xfVPQbxWhbD2MYu6EOXEoi3QjHkjIA7qOdIUo4H0vl8jW70Ga3cJIa-y7Hs7LTQSdFXRu-znCdKjSuaiE8HP9aUdw Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:09:55.484681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_66c7f61afd0d4650bd530eff18a2318b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: c4a002047e3e42338e4f57ba49e877fb Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.487434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'server_id': 'd86c18ba-e366-4ccb-823b-d3c453e49d6c'} Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:09:55.488875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:55.491886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:55.494676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Returning 404 to user: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:09:55.494676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3303dc3d-64a0-415f-aafe-891e7e1243fd tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.058801 Jul 31 01:09:55.494676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 85/172] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:09:55 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c/os-volume_attachments => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:09:57.198296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:57.199130 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:57.223173 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:57.224325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:57.234382 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:57.235487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:57.273772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] HTTP exception thrown: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Request method failure captured: Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: DELETE /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c HTTP/1.1 Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dX7IQPomRQzRAIfEJmiewWpdR29NMFQSclNoB4VWMYfaacPRuQGX-SE70k1cvEKHjfRUxq60oQj1emNQ8BJG6xfVPQbxWhbD2MYu6EOXEoi3QjHkjIA7qOdIUo4H0vl8jW70Ga3cJIa-y7Hs7LTQSdFXRu-znCdKjSuaiE8HP9aUdw Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:09:57.283466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_66c7f61afd0d4650bd530eff18a2318b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: c4a002047e3e42338e4f57ba49e877fb Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.286293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'd86c18ba-e366-4ccb-823b-d3c453e49d6c'} Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:09:57.287818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.290631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.298167 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Returning 404 to user: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:09:57.304898 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cb86500f-77ec-4ac8-a7d8-702dc65b31e4 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 404 len: 111 microversion: 2.1 time: 0.110024 Jul 31 01:09:57.306881 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 88/173] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:09:57 2022] DELETE /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 111 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:09:57.319400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:09:57.325814 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] [instance: d86c18ba-e366-4ccb-823b-d3c453e49d6c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:09:57.346791 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:57.346791 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:57.348964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:09:57.351582 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:09:57.398478 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] HTTP exception thrown: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Request method failure captured: Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c HTTP/1.1 Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dX7IQPomRQzRAIfEJmiewWpdR29NMFQSclNoB4VWMYfaacPRuQGX-SE70k1cvEKHjfRUxq60oQj1emNQ8BJG6xfVPQbxWhbD2MYu6EOXEoi3QjHkjIA7qOdIUo4H0vl8jW70Ga3cJIa-y7Hs7LTQSdFXRu-znCdKjSuaiE8HP9aUdw Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:09:57.402770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/66c7f61afd0d4650bd530eff18a2318b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_66c7f61afd0d4650bd530eff18a2318b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 66c7f61afd0d4650bd530eff18a2318b Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-DeleteServersTestJSON-1032424423 Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: c4a002047e3e42338e4f57ba49e877fb Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-DeleteServersTestJSON-1032424423-project Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.405987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'd86c18ba-e366-4ccb-823b-d3c453e49d6c'} Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:09:57.407523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. Jul 31 01:09:57.409751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:09:57.416279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] Returning 404 to user: Instance d86c18ba-e366-4ccb-823b-d3c453e49d6c could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:09:57.416279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-31560047-0aff-452c-807a-adf129d50b26 tempest-DeleteServersTestJSON-1032424423 tempest-DeleteServersTestJSON-1032424423-project] 10.222.0.42 "GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c" status: 404 len: 111 microversion: 2.1 time: 0.102579 Jul 31 01:09:57.416636 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 86/174] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:09:57 2022] GET /compute/v2.1/servers/d86c18ba-e366-4ccb-823b-d3c453e49d6c => generated 111 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:10:07.323397 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-376113013", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "networks": [{"uuid": "017f6b73-bc7c-4af9-b8ed-06704cf8963d"}]}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:07.346747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=12 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:07.349309 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=28 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:07.441995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] validate_networks() for [('017f6b73-bc7c-4af9-b8ed-06704cf8963d', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:10:08.226911 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:10:08.227665 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:10:08.228526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:10:08.229262 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:10:08.229902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:10:08.278470 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:08.283423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:08.288697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:08.289745 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:08.362657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Getting quotas for project de83743a559b4a289aa40c8d16641ee9. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:10:08.391438 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Getting quotas for user 8fcc34c42bb64200ac5f40d770d6784b and project de83743a559b4a289aa40c8d16641ee9. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:10:08.421980 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:10:08.432484 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:10:08.541969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a4b0696d-7aa9-4fa4-a665-22295313b0cd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.227310 Jul 31 01:10:08.552211 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 89/175] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:10:07 2022] POST /compute/v2.1/servers => generated 380 bytes in 1239 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:10:08.565035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a37eefdd-7b2b-49eb-a22b-b3a5f492df6a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:08.567410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a37eefdd-7b2b-49eb-a22b-b3a5f492df6a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:08.578122 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=21,nova_api:UPDATE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:08.579347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:08.780278 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a37eefdd-7b2b-49eb-a22b-b3a5f492df6a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:08.829237 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a37eefdd-7b2b-49eb-a22b-b3a5f492df6a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1213 microversion: 2.1 time: 0.272810 Jul 31 01:10:08.870741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 87/176] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:08 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1213 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:09.880612 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:09.882055 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:09.896162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:09.896870 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:09.898272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:09.899272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:10.009296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:10.022789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e27fc033-d3a2-467b-937a-f2dc35a4798a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1217 microversion: 2.1 time: 0.154843 Jul 31 01:10:10.023848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 90/177] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:09 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1217 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:11.075653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:11.076789 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:11.101591 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:11.105347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:11.106716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:11.108038 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:11.234007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:11.250439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-327e787f-2f46-4f9a-beb3-221b4b44f361 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1209 microversion: 2.1 time: 0.194504 Jul 31 01:10:11.252600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 88/178] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:11 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1209 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:12.290158 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:12.291341 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:12.321252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:12.322484 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:12.324388 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:12.331910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:12.546347 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:12.558726 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-65bd7730-a649-47e8-8fd2-b55063679b43 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1273 microversion: 2.1 time: 0.274097 Jul 31 01:10:12.560658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 91/179] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:12 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1273 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:13.586012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:13.589356 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:13.651018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:13.653936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:13.667704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:13.673958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:14.167389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:14.190040 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-36ca20e4-b7cf-467c-a024-b43ae5688d83 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1271 microversion: 2.1 time: 0.611753 Jul 31 01:10:14.212432 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 89/180] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:13 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1271 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:15.249566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:15.254600 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:15.270001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:15.271318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:15.273246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:15.275162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:15.742510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:15.777300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5f650a61-9b87-465e-aedc-785728711499 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.548149 Jul 31 01:10:15.779424 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 92/181] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:15 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:15.786091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2049263765", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:15.902615 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] validate_networks() for [('920eee56-6f5f-4984-b783-e122bc773edb', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:10:16.228037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-27187e01-a0a7-46fc-a151-183ce8c96d24 req-c20f1778-2118-4df8-a7af-36f1e4a6000d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3433ac58-bcf0-4ba2-b145-a88f99cb5a7c", "tag": "41439ba7-7620-4a6d-8b5d-4fe31b15e44f"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:16.292782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-27187e01-a0a7-46fc-a151-183ce8c96d24 req-c20f1778-2118-4df8-a7af-36f1e4a6000d service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:16.299704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-27187e01-a0a7-46fc-a151-183ce8c96d24 req-c20f1778-2118-4df8-a7af-36f1e4a6000d service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:16.383753 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-27187e01-a0a7-46fc-a151-183ce8c96d24 req-c20f1778-2118-4df8-a7af-36f1e4a6000d service nova] Creating event network-changed:41439ba7-7620-4a6d-8b5d-4fe31b15e44f for instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c on hv-cinder-maste Jul 31 01:10:16.456428 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-27187e01-a0a7-46fc-a151-183ce8c96d24 req-c20f1778-2118-4df8-a7af-36f1e4a6000d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.253916 Jul 31 01:10:16.461827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 93/182] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:10:16.808657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:16.810047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:16.836995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:16.837874 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:16.846375 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:16.847344 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:17.161367 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:17.179432 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f68eb853-b8e8-4dc1-a692-ef3c446b4e15 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.379093 Jul 31 01:10:17.181054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 94/183] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:16 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:17.798722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:10:17.799263 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:10:17.799817 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:10:17.800353 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:10:17.800942 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:10:17.821864 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:17.823130 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:17.830179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:17.830179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:17.869788 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:10:17.880936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for user 52e5a4deed99442e8eed62f334aa670d and project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:10:17.907378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:10:17.913263 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:10:18.024666 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-37851089-13db-4007-ad87-4582e8e5498e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.639357 Jul 31 01:10:18.026147 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 90/184] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:10:15 2022] POST /compute/v2.1/servers => generated 383 bytes in 2643 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 31 01:10:18.042195 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-49ed6943-0ce0-4e3b-82d6-7474d03c61bc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:18.044259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-49ed6943-0ce0-4e3b-82d6-7474d03c61bc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:18.196445 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-49ed6943-0ce0-4e3b-82d6-7474d03c61bc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:18.220221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:18.223464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:18.223739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-49ed6943-0ce0-4e3b-82d6-7474d03c61bc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1222 microversion: 2.1 time: 0.187165 Jul 31 01:10:18.226681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 95/185] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:18 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1222 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:18.251191 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:18.254537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:18.257868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:18.259063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:18.583752 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1088852709", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:18.614949 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:18.644356 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:18.686421 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-26c2312e-3b17-471f-acf9-23fce0ac9213 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.469301 Jul 31 01:10:18.698746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 91/186] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:18 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:18.762234 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] validate_networks() for [('920eee56-6f5f-4984-b783-e122bc773edb', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:10:19.740528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:19.741102 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:19.766719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:19.773667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:19.774526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:19.774978 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:19.785845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:10:19.787828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:10:19.789085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:10:19.790070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:10:19.791595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:10:19.815222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:19.816546 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:19.824944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:19.826602 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:19.877544 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:10:19.888133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for user 52e5a4deed99442e8eed62f334aa670d and project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:10:19.930716 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:10:19.939961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:10:20.056312 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b0b26dd5-16c4-4454-9bb7-516792f86067 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.486372 Jul 31 01:10:20.057358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 96/187] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:10:18 2022] POST /compute/v2.1/servers => generated 383 bytes in 1489 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:10:20.085154 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-311a12c1-61c7-47cb-942e-9d5cc21e8154 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:20.086090 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-311a12c1-61c7-47cb-942e-9d5cc21e8154 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:20.238291 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-311a12c1-61c7-47cb-942e-9d5cc21e8154 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:20.254538 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-311a12c1-61c7-47cb-942e-9d5cc21e8154 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1222 microversion: 2.1 time: 0.184024 Jul 31 01:10:20.256000 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 97/188] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:20 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1222 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:20.270517 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:20.282007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ca65e864-9ee7-40f7-8cb9-0da8a3ec7491 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.550502 Jul 31 01:10:20.293125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 92/189] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:19 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:21.292801 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:21.295119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:21.311972 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:21.312777 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:21.320822 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:21.321718 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:21.323293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:21.324927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:21.328321 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:21.329208 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:21.330438 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:21.333218 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:21.544360 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:21.566415 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ed0c3eaf-524f-475f-9dd8-dde398428ca8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1226 microversion: 2.1 time: 0.274930 Jul 31 01:10:21.568776 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 98/190] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:21 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1226 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:21.744241 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:21.759060 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-961b8ed6-ee9f-4170-8b85-ac1c69303a98 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.454718 Jul 31 01:10:21.775520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 93/191] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:21 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:22.606927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:22.611186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:22.620894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:22.621555 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:22.622447 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:22.622944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:22.708254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:22.719900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-855009ba-256f-4a24-9040-a5a91c6d08aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1226 microversion: 2.1 time: 0.119299 Jul 31 01:10:22.720736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 99/192] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:22 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1226 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:22.799731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:22.802531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:22.810813 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:22.811529 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:22.812475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:22.813071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:23.031331 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:23.045987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-df22f89e-4e51-4636-947f-7dad99aa0ddb tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.253990 Jul 31 01:10:23.046866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 94/193] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:22 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:23.765861 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:23.766764 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:23.774754 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:23.775396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:23.776370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:23.776970 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:23.911566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:23.921250 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-260c24f3-dfee-4bd7-a9f6-65b52ea09e2b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1226 microversion: 2.1 time: 0.160245 Jul 31 01:10:23.922104 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 100/194] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:23 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1226 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:24.071640 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:24.072688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:24.084600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:24.085415 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:24.088014 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:24.088748 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:24.369847 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:24.400586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-af4222e8-fa00-4ce5-997f-9842e50c444e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.334715 Jul 31 01:10:24.401437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 95/195] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:24 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:24.957704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:24.958748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:24.965997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:24.967263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:24.968139 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:24.968737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:25.068020 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:25.079173 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9f44efbe-6766-4300-8f9f-42599031c4e3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1274 microversion: 2.1 time: 0.127255 Jul 31 01:10:25.080273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 101/196] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:24 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1274 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:25.440499 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:25.451801 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:25.488520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:25.490221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:25.493680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:25.501325 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:26.131710 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:26.134730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:26.157552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:26.159602 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:26.161767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:26.163550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:26.218085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:26.235747 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-961a37f0-a9cf-4d0f-b148-924e102d13f0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.804441 Jul 31 01:10:26.237134 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 96/197] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:25 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 807 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:26.298521 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:26.313855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-dd49b2d9-380c-4bc9-8fdb-f174e5560cf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1280 microversion: 2.1 time: 0.202157 Jul 31 01:10:26.315662 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 102/198] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:26 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1280 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:27.263726 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:27.264944 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:27.287304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:27.288473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:27.289944 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:27.290992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:27.354914 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:27.358741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:27.395143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:27.398664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:27.407427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:27.416651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:27.850929 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:27.869608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:27.900095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=8 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:27.928287 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-bebfad82-ce82-4893-9c9c-5c0f1a42ee5e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.670636 Jul 31 01:10:27.929754 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 97/199] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:27 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 675 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:27.988600 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:28.021677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bdacf972-267d-4c2a-a007-65abdbfe330b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.686270 Jul 31 01:10:28.022683 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 103/200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:27 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:28.300573 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-ada8702e-8666-43f6-9c25-08c13d17b70b req-a1d650b8-44d5-46be-b668-df1702d87929 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3433ac58-bcf0-4ba2-b145-a88f99cb5a7c", "name": "network-vif-plugged", "status": "completed", "tag": "41439ba7-7620-4a6d-8b5d-4fe31b15e44f"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:28.378682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ada8702e-8666-43f6-9c25-08c13d17b70b req-a1d650b8-44d5-46be-b668-df1702d87929 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:28.386087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ada8702e-8666-43f6-9c25-08c13d17b70b req-a1d650b8-44d5-46be-b668-df1702d87929 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:28.430884 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-ada8702e-8666-43f6-9c25-08c13d17b70b req-a1d650b8-44d5-46be-b668-df1702d87929 service nova] Creating event network-vif-plugged:41439ba7-7620-4a6d-8b5d-4fe31b15e44f for instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c on hv-cinder-maste Jul 31 01:10:28.452106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-ada8702e-8666-43f6-9c25-08c13d17b70b req-a1d650b8-44d5-46be-b668-df1702d87929 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.161229 Jul 31 01:10:28.456120 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 98/201] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:10:28.744380 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-42f10d01-6324-4a60-95f4-0b33ea57ea2f req-fac9c92c-accc-4343-aa5f-3461054f49a6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d2206492-6f7a-4a00-ac23-63200f66c940", "tag": "78c48998-1895-47eb-bd4c-6c4a5145f1e5"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:28.787339 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-42f10d01-6324-4a60-95f4-0b33ea57ea2f req-fac9c92c-accc-4343-aa5f-3461054f49a6 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:28.788567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-42f10d01-6324-4a60-95f4-0b33ea57ea2f req-fac9c92c-accc-4343-aa5f-3461054f49a6 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:28.828314 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-42f10d01-6324-4a60-95f4-0b33ea57ea2f req-fac9c92c-accc-4343-aa5f-3461054f49a6 service nova] Creating event network-changed:78c48998-1895-47eb-bd4c-6c4a5145f1e5 for instance d2206492-6f7a-4a00-ac23-63200f66c940 on hv-cinder-maste Jul 31 01:10:28.840117 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-42f10d01-6324-4a60-95f4-0b33ea57ea2f req-fac9c92c-accc-4343-aa5f-3461054f49a6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102098 Jul 31 01:10:28.842480 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 104/202] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:10:28.959705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:28.960541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:28.970574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:28.971643 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:28.972982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:28.973622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:29.060104 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:29.060995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:29.067789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:29.068512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:29.069540 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:29.070011 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:29.257860 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:29.275259 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5169f67a-fbb2-4dbf-8110-6e9c77209d21 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.320133 Jul 31 01:10:29.292322 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 99/203] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:28 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:29.378300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:29.391126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9f396b19-2779-474b-a9e2-fcacd97bf1de tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.335675 Jul 31 01:10:29.392186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 105/204] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:29 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:29.392733 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-1b5ebd12-6aa1-414c-a646-dd2a05405e84 req-dfd3b108-472d-4b69-bb94-719e34241e65 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "973d94b6-5c7b-4acf-a633-8ed031945845", "tag": "05db22e9-ff42-4276-93e4-b10ebcebcb40"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:29.418774 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1b5ebd12-6aa1-414c-a646-dd2a05405e84 req-dfd3b108-472d-4b69-bb94-719e34241e65 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:29.421167 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1b5ebd12-6aa1-414c-a646-dd2a05405e84 req-dfd3b108-472d-4b69-bb94-719e34241e65 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:29.457163 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-1b5ebd12-6aa1-414c-a646-dd2a05405e84 req-dfd3b108-472d-4b69-bb94-719e34241e65 service nova] Creating event network-changed:05db22e9-ff42-4276-93e4-b10ebcebcb40 for instance 973d94b6-5c7b-4acf-a633-8ed031945845 on hv-cinder-maste Jul 31 01:10:29.464549 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-1b5ebd12-6aa1-414c-a646-dd2a05405e84 req-dfd3b108-472d-4b69-bb94-719e34241e65 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082663 Jul 31 01:10:29.465517 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 100/205] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:10:30.335471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:30.339760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:30.368569 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:30.370737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=16 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:30.382176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:30.383853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:30.385654 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:30.386794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:30.435759 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:30.435759 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:30.454147 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:30.454623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:30.455509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:30.455960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:30.729768 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:30.753835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bde42619-ccad-412e-9372-a7a9c79a3641 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.432112 Jul 31 01:10:30.764737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 106/206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:30 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:30.940814 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:30.953260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a1ff3a7b-cfa5-4a7d-af05-d8d3030ad7b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.523113 Jul 31 01:10:30.954585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 101/207] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:30 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:31.782448 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:31.783094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:31.790111 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:31.790564 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:31.791374 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:31.791838 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:31.949485 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:31.962977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8ca6ecf3-61fd-4635-876d-982b52a15355 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.184131 Jul 31 01:10:31.964082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 107/208] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:31 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:31.990855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:31.992081 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:32.002439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:32.003063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:32.003966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:32.004514 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:32.177256 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:32.186633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1272f185-7082-4166-be49-9f8cd11df486 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.201077 Jul 31 01:10:32.187260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 102/209] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:31 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:32.985449 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:32.986273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:32.993590 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:32.994151 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:32.995087 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:32.995634 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:33.183553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:33.196577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-75ce891f-5285-406c-9bc8-51a6415e4f1b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1313 microversion: 2.1 time: 0.215489 Jul 31 01:10:33.197248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 108/210] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:32 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1313 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:33.210692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:33.211683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:33.220564 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:33.221139 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:33.222038 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:33.222523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:33.412813 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:33.426307 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e2149640-ced3-4682-a5bb-88a700693be8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.220929 Jul 31 01:10:33.427405 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 103/211] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:33 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:34.234124 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:34.237663 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:34.250288 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:34.251669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:34.253458 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:34.254659 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:34.469821 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:34.472189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:34.499662 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:34.501212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:34.503207 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:34.504331 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:34.917727 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:34.933833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7cf941f7-632d-4176-ac4f-131718db5389 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1497 microversion: 2.1 time: 0.708129 Jul 31 01:10:34.935493 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 109/212] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:34 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1497 bytes in 711 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:34.998503 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:35.028052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cd14c040-ecfe-4522-856a-4d86561c76ed tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.568199 Jul 31 01:10:35.029170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 104/213] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:34 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:35.801657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:35.802552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:35.815678 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:35.816459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:35.818070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:35.818852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:35.828266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:35.879227 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:10:36.036997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-978e6c86-72cb-44d1-a2cf-7417e85f3fc2 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 204 len: 0 microversion: 2.1 time: 0.242097 Jul 31 01:10:36.038691 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 110/214] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:10:35 2022] DELETE /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:10:36.051396 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:36.052461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:36.062728 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:36.064115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:36.065149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:36.066240 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:36.068092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:36.070089 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:36.081681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:36.081681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:36.081681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:36.081681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:36.338694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:36.345135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:36.350827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-055535a7-dc9e-4299-9b78-22e60b8c1376 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.296089 Jul 31 01:10:36.351961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 111/215] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:36 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:36.363515 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2389f3fc-8c97-47ac-b4ed-8cb67a9087f1 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1503 microversion: 2.1 time: 0.318066 Jul 31 01:10:36.365506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 105/216] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:36 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1503 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:37.386938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:37.400850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:37.402760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:37.411601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:37.412370 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:37.412980 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:37.414644 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:37.415272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:37.428226 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:37.444648 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:37.447245 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:37.451559 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:37.454258 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:37.455818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:37.762054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:37.774862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c289e739-3bc0-43cd-9753-853db9710aab tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1503 microversion: 2.1 time: 0.384801 Jul 31 01:10:37.784219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 106/217] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:37 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1503 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:37.809511 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:37.818199 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-91a4732f-06f9-4736-8c5c-b8e869b55615 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.440227 Jul 31 01:10:37.818875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 112/218] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:37 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:38.025439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-f7da7d65-92ae-44cc-add3-7cac9e11f58f req-a19238aa-0f14-4559-af89-c94005e86f2a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d2206492-6f7a-4a00-ac23-63200f66c940", "name": "network-vif-plugged", "status": "completed", "tag": "78c48998-1895-47eb-bd4c-6c4a5145f1e5"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:38.042851 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f7da7d65-92ae-44cc-add3-7cac9e11f58f req-a19238aa-0f14-4559-af89-c94005e86f2a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:38.044494 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f7da7d65-92ae-44cc-add3-7cac9e11f58f req-a19238aa-0f14-4559-af89-c94005e86f2a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:38.066607 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-f7da7d65-92ae-44cc-add3-7cac9e11f58f req-a19238aa-0f14-4559-af89-c94005e86f2a service nova] Creating event network-vif-plugged:78c48998-1895-47eb-bd4c-6c4a5145f1e5 for instance d2206492-6f7a-4a00-ac23-63200f66c940 on hv-cinder-maste Jul 31 01:10:38.073912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-f7da7d65-92ae-44cc-add3-7cac9e11f58f req-a19238aa-0f14-4559-af89-c94005e86f2a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055314 Jul 31 01:10:38.074922 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 107/219] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:10:38.798400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:38.799094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:38.807019 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:38.807453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:38.808609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:38.809145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:38.842605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:38.843918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:38.848670 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:38.856988 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:38.858257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:38.859749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:38.860890 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:39.020489 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:39.035740 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a09ae2d1-ab43-47a5-a79d-1822dc808555 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1503 microversion: 2.1 time: 0.241540 Jul 31 01:10:39.037085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 113/220] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:38 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1503 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:39.128110 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:39.164769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d86bada8-82ad-475c-ad42-c08398323d08 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.327891 Jul 31 01:10:39.166053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 108/221] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:38 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:40.058369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:40.060033 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:40.069260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:40.070009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:40.071103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:40.071685 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:40.201707 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:40.208520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:40.224335 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:40.225390 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:40.226900 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:40.228034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:40.236472 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:40.259022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7f29c1a4-201f-40ee-9f2c-caa69df88f41 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1461 microversion: 2.1 time: 0.204834 Jul 31 01:10:40.265147 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 114/222] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:40 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1461 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:40.280191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-315a87c9-26c4-4fe6-b2c6-f01809c39c1f req-3796e5ae-525f-46bc-b5ca-c2d265b2e255 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3433ac58-bcf0-4ba2-b145-a88f99cb5a7c", "name": "network-vif-deleted", "tag": "41439ba7-7620-4a6d-8b5d-4fe31b15e44f"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:40.299160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-315a87c9-26c4-4fe6-b2c6-f01809c39c1f req-3796e5ae-525f-46bc-b5ca-c2d265b2e255 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:40.300364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-315a87c9-26c4-4fe6-b2c6-f01809c39c1f req-3796e5ae-525f-46bc-b5ca-c2d265b2e255 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:40.323886 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-315a87c9-26c4-4fe6-b2c6-f01809c39c1f req-3796e5ae-525f-46bc-b5ca-c2d265b2e255 service nova] Creating event network-vif-deleted:41439ba7-7620-4a6d-8b5d-4fe31b15e44f for instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c on hv-cinder-maste Jul 31 01:10:40.334473 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-315a87c9-26c4-4fe6-b2c6-f01809c39c1f req-3796e5ae-525f-46bc-b5ca-c2d265b2e255 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063818 Jul 31 01:10:40.336703 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 115/223] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:10:40.472454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:40.487315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9bf3e3ac-d059-4172-8539-9a36c0cee40b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.291848 Jul 31 01:10:40.488228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 109/224] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:40 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:41.298062 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:41.298849 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:41.305717 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:41.306218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:41.307081 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:41.307663 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:41.351731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:41.352319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:41.352889 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lazy-loading 'fault' on Instance uuid 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:10:41.408216 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:41.418550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a438d687-5822-4820-94fa-fd8ca099c501 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1303 microversion: 2.1 time: 0.125635 Jul 31 01:10:41.420216 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 116/225] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:41 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1303 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:41.524949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:41.525985 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:41.536029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:41.536543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:41.537380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:41.537861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:41.707280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-736308497", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:41.717477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:41.726669 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-54799d4a-ef0f-46e5-af22-2363619543eb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.207082 Jul 31 01:10:41.727275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 110/226] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:41 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:41.768708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] validate_networks() for None {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:10:42.120544 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-e28b1bff-0405-4e08-9970-8cd0e66ca942 req-fefcc658-81d3-40ec-8ada-e48e97a314c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "973d94b6-5c7b-4acf-a633-8ed031945845", "name": "network-vif-plugged", "status": "completed", "tag": "05db22e9-ff42-4276-93e4-b10ebcebcb40"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:42.143983 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-e28b1bff-0405-4e08-9970-8cd0e66ca942 req-fefcc658-81d3-40ec-8ada-e48e97a314c2 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.145440 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-e28b1bff-0405-4e08-9970-8cd0e66ca942 req-fefcc658-81d3-40ec-8ada-e48e97a314c2 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:42.170273 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-e28b1bff-0405-4e08-9970-8cd0e66ca942 req-fefcc658-81d3-40ec-8ada-e48e97a314c2 service nova] Creating event network-vif-plugged:05db22e9-ff42-4276-93e4-b10ebcebcb40 for instance 973d94b6-5c7b-4acf-a633-8ed031945845 on hv-cinder-maste Jul 31 01:10:42.178868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-e28b1bff-0405-4e08-9970-8cd0e66ca942 req-fefcc658-81d3-40ec-8ada-e48e97a314c2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064912 Jul 31 01:10:42.180217 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 111/227] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:10:42.190847 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:10:42.191436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:10:42.192211 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:10:42.193466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:10:42.193900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:10:42.206323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.207089 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:42.208290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.209917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:42.234765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Getting quotas for project bf5ca398012441cba969788634f537b8. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:10:42.239683 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Getting quotas for user 2e69ff835a0247869749c889143d8275 and project bf5ca398012441cba969788634f537b8. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:10:42.253496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:10:42.256208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:10:42.316376 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e14ad9d6-9600-4870-9597-cfd843c7cbdf tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.613140 Jul 31 01:10:42.317517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 117/228] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:10:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 615 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 31 01:10:42.337003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2de4209a-e6e3-4dc2-a554-1423d08cf4cd tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:42.338808 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2de4209a-e6e3-4dc2-a554-1423d08cf4cd tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:42.446220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:42.447338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:42.465264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2de4209a-e6e3-4dc2-a554-1423d08cf4cd tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:42.479587 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2de4209a-e6e3-4dc2-a554-1423d08cf4cd tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1214 microversion: 2.1 time: 0.148927 Jul 31 01:10:42.481481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.482380 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:42.482850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 112/229] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:42 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1214 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:42.495615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.495615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:42.577494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.578035 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:42.578543 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lazy-loading 'fault' on Instance uuid 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:10:42.662124 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:42.678203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a3d231d0-5cf4-4c75-9439-f8fe91bdcb5a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 200 len: 1303 microversion: 2.1 time: 0.239203 Jul 31 01:10:42.679518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 118/230] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:42 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 1303 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:42.759723 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:42.760580 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:42.770299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.771071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:42.772802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:42.773953 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:43.016240 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:43.032691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-17c20dbc-1015-4c05-b7f5-6530f45490ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.277414 Jul 31 01:10:43.033848 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 113/231] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:42 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:43.523502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:43.524241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:43.532177 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:43.532632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:43.533417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:43.533854 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:43.615567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:43.625702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-29d95774-1d42-4e29-93d8-8be2a3cc44b0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1210 microversion: 2.1 time: 0.106202 Jul 31 01:10:43.626436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 119/232] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:43 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1210 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:43.707823 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:43.708491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:43.718029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:43.718701 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:43.719535 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:43.720019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:43.741005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] HTTP exception thrown: Instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c could not be found. Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Request method failure captured: Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c HTTP/1.1 Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dZrdLW10-XzTesHhUghgJp-iRJC_pyNwHJoAzstK9NY9Kjc5nxm7NUbHv2P4U_LQfq5nI2id6MYONtowHj2QUy2x6xH64Wy1DJo1GNcXJi4v4oxetOAgJex_7HVeLOx6PN52FV1PLWOi9fD8G-3pXvreD-F4kelhe_elibIreX9fGk Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: de83743a559b4a289aa40c8d16641ee9 Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-VolumesActionsTest-1592492359 Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:10:43.743423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/de83743a559b4a289aa40c8d16641ee9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/de83743a559b4a289aa40c8d16641ee9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/de83743a559b4a289aa40c8d16641ee9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_de83743a559b4a289aa40c8d16641ee9", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-VolumesActionsTest-1592492359 Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: de83743a559b4a289aa40c8d16641ee9 Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-VolumesActionsTest-1592492359 Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-VolumesActionsTest-1592492359-project Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 8fcc34c42bb64200ac5f40d770d6784b Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-VolumesActionsTest-1592492359-project Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c could not be found. Jul 31 01:10:43.745819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '3433ac58-bcf0-4ba2-b145-a88f99cb5a7c'} Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c could not be found. Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:10:43.747170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c could not be found. Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c could not be found. Jul 31 01:10:43.749508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:10:43.751958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Returning 404 to user: Instance 3433ac58-bcf0-4ba2-b145-a88f99cb5a7c could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:10:43.751958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5316c470-7fc2-48f6-bd8c-3dba48d26286 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c" status: 404 len: 111 microversion: 2.1 time: 0.041912 Jul 31 01:10:43.751958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 114/233] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:43 2022] GET /compute/v2.1/servers/3433ac58-bcf0-4ba2-b145-a88f99cb5a7c => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:10:43.905366 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1198159731", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "networks": [{"uuid": "017f6b73-bc7c-4af9-b8ed-06704cf8963d"}]}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:43.973109 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] validate_networks() for [('017f6b73-bc7c-4af9-b8ed-06704cf8963d', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:10:44.066491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:44.067482 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:44.077815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:44.079464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:44.079464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:44.079840 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:44.348230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:44.364238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e1964a24-0f4c-4fb2-be5a-2193e55c3357 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1322 microversion: 2.1 time: 0.304348 Jul 31 01:10:44.365288 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 115/234] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:44 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1322 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:44.531279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:10:44.532073 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:10:44.532248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:10:44.532682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:10:44.533091 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:10:44.543882 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:44.544491 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:44.545317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:44.545784 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:44.565955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Getting quotas for project de83743a559b4a289aa40c8d16641ee9. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:10:44.570611 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Getting quotas for user 8fcc34c42bb64200ac5f40d770d6784b and project de83743a559b4a289aa40c8d16641ee9. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:10:44.584323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:10:44.586586 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:10:44.636872 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-71527038-3cd4-4624-8ce7-d2412aa3121e tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.736642 Jul 31 01:10:44.648589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 120/235] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:10:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 739 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:10:44.654123 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:44.654706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d02ac5b3-7609-48c0-a72b-ea63f2a23811 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:44.655246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:44.655575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d02ac5b3-7609-48c0-a72b-ea63f2a23811 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:44.677477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:44.678096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:44.679112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:44.679769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:44.755397 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d02ac5b3-7609-48c0-a72b-ea63f2a23811 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:44.769433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d02ac5b3-7609-48c0-a72b-ea63f2a23811 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1214 microversion: 2.1 time: 0.119286 Jul 31 01:10:44.770450 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 121/236] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:44 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1214 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:44.794144 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:44.808460 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9eb519a5-58bf-492f-8699-adaaee3963b2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1266 microversion: 2.1 time: 0.162030 Jul 31 01:10:44.809850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 116/237] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:44 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1266 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:45.392028 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:45.393000 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:45.406021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.406828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:45.408565 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.409365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:45.697076 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:45.716588 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b60b2109-2f9f-4eda-abcd-610b58c1f9d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1517 microversion: 2.1 time: 0.330321 Jul 31 01:10:45.719174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 122/238] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:45 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1517 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:45.741005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:45.744484 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:45.753299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.754240 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:45.755253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.755928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:45.799157 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:45.800083 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:45.801325 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lazy-loading 'flavor' on Instance uuid 973d94b6-5c7b-4acf-a633-8ed031945845 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:10:45.818246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.819219 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:45.820898 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.821893 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:45.928194 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:45.942463 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-000d5cbb-c283-4eac-a825-d0a5b2e713b3 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1266 microversion: 2.1 time: 0.148426 Jul 31 01:10:45.943857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 123/239] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:45 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1266 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:45.953172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:45.954357 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:45.968832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.969388 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:45.970290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:45.970748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:46.056202 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:46.062791 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-76014d6d-ef73-40be-b71b-0c00cda93484 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845/action" status: 200 len: 32 microversion: 2.1 time: 0.326576 Jul 31 01:10:46.063440 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 117/240] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:10:45 2022] POST /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845/action => generated 32 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:10:46.072021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f667c484-8288-4cf0-bd39-c690043588c2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1272 microversion: 2.1 time: 0.126892 Jul 31 01:10:46.072943 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 124/241] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:45 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1272 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:46.082941 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:46.084551 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:46.096874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:46.097704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:46.099146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:46.099987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:46.645995 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:46.661892 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3b0cc508-736d-4dce-be68-37b104b5727f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.588399 Jul 31 01:10:46.675398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 118/242] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:46 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:46.981673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:46.982572 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:46.991039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:46.991581 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:46.992438 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:46.992914 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:47.094336 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:47.095693 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:47.103918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:47.104859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:47.106061 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:47.106787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:47.328857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:47.336860 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7d5ac020-629e-40fb-9727-e9c3d4115e63 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1272 microversion: 2.1 time: 0.361002 Jul 31 01:10:47.343985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 125/243] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:46 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1272 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:47.690775 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:47.705859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:47.706513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0bc7eae8-f25b-430f-9253-5776fed65972 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.616045 Jul 31 01:10:47.707574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 119/244] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:47 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:47.721936 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:47.735461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:47.736566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:47.737757 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:47.738840 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:48.057203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:48.069202 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-907c6fc2-b1d1-4fe0-a56a-81f43fdc0ddd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.374417 Jul 31 01:10:48.069932 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 126/245] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:47 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:48.180230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-f8848004-642e-44b2-b35f-116815016129 req-37b9ff01-8cc6-4a36-a66d-66c28d7e4a06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1fc0a86-e494-4707-bac6-ef80822d851b", "tag": "586fdcee-a89e-4e95-9f02-c485e653607f"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:48.199907 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f8848004-642e-44b2-b35f-116815016129 req-37b9ff01-8cc6-4a36-a66d-66c28d7e4a06 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:48.200567 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f8848004-642e-44b2-b35f-116815016129 req-37b9ff01-8cc6-4a36-a66d-66c28d7e4a06 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:48.232032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-f8848004-642e-44b2-b35f-116815016129 req-37b9ff01-8cc6-4a36-a66d-66c28d7e4a06 service nova] Creating event network-changed:586fdcee-a89e-4e95-9f02-c485e653607f for instance a1fc0a86-e494-4707-bac6-ef80822d851b on hv-cinder-maste Jul 31 01:10:48.240565 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-f8848004-642e-44b2-b35f-116815016129 req-37b9ff01-8cc6-4a36-a66d-66c28d7e4a06 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067466 Jul 31 01:10:48.241636 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 120/246] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:10:48.363934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:48.365134 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:48.375163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:48.375954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:48.377018 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:48.377558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:48.400714 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:48.737731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:48.738477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:48.746488 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:48.746978 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:48.748052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:48.748502 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:48.788406 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:48.804951 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-54b4e4a6-39c5-4606-9dde-d525ed8a6c26 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.448811 Jul 31 01:10:48.806917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 127/247] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:48 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:49.017330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:49.033125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-82fc2869-c02f-4c92-935b-1fb1e970dc30 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.299311 Jul 31 01:10:49.034682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 121/248] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:48 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:49.102689 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:49.103433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:49.112357 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:49.113123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:49.114435 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:49.115060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:49.310515 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-69f9fa4d-2fd3-4590-adf6-54f13972f4e5 req-0345ece1-ce1d-4a44-8171-a5f968251ab8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5", "tag": "a472a1b9-6f91-4385-b80e-65adbe320d3f"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:49.326480 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-69f9fa4d-2fd3-4590-adf6-54f13972f4e5 req-0345ece1-ce1d-4a44-8171-a5f968251ab8 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:49.327270 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-69f9fa4d-2fd3-4590-adf6-54f13972f4e5 req-0345ece1-ce1d-4a44-8171-a5f968251ab8 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:49.335300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:49.346707 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-69f9fa4d-2fd3-4590-adf6-54f13972f4e5 req-0345ece1-ce1d-4a44-8171-a5f968251ab8 service nova] Creating event network-changed:a472a1b9-6f91-4385-b80e-65adbe320d3f for instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 on hv-cinder-maste Jul 31 01:10:49.349274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-61b3feff-d61d-45a4-9875-a031b5846325 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.250661 Jul 31 01:10:49.359066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 128/249] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:49 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:49.367258 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-69f9fa4d-2fd3-4590-adf6-54f13972f4e5 req-0345ece1-ce1d-4a44-8171-a5f968251ab8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064575 Jul 31 01:10:49.368292 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 122/250] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:10:49.834702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:49.835223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:49.844495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:49.845285 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:49.846643 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:49.847461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:50.048087 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:50.056347 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-256811a0-292a-4bda-9411-74993bf8217a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.225820 Jul 31 01:10:50.056956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 129/251] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:49 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:50.071925 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:50.072805 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:50.080972 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:50.081298 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:50.082200 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:50.082654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:50.256891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:50.266461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-75f9040f-ec16-4d5d-8c72-a37191d637dc tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.199263 Jul 31 01:10:50.267252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 123/252] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:50 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:50.373430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:50.374438 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:50.385453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:50.386246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:50.387731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:50.388566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:50.556298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:50.565548 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c66bdabc-ef7d-450f-99ef-addb25f39336 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.196826 Jul 31 01:10:50.566323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 130/253] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:50 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:51.079513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:51.080188 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:51.087012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:51.087509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:51.088314 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:51.088777 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:51.240600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:51.251400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f4e18eb1-8e26-4981-926c-a705b22d2a3b tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.175904 Jul 31 01:10:51.251969 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 124/254] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:51 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:51.290741 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:51.291784 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:51.302900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:51.303565 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:51.304771 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:51.305449 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:51.470987 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:51.479791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d19d58c3-0a75-4092-aa53-d9cb92be2d93 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.194956 Jul 31 01:10:51.480402 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 131/255] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:51 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:51.595725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:51.596519 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:51.603290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:51.603786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:51.604573 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:51.605018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:51.626668 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:51.769793 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:51.785288 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2d86d633-d6ca-4d51-bf9b-7e0cb0a13824 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.193562 Jul 31 01:10:51.786302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 125/256] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:51 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:52.272886 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:52.273580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:52.280603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:52.281054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:52.281996 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:52.282436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:52.441573 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:52.450463 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1583dbe0-60b0-4fb4-8995-45848373d1fe tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.181502 Jul 31 01:10:52.451076 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 132/257] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:52 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:52.502585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:52.503726 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:52.506153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:52.512892 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:52.513422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:52.514496 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:52.515068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:52.673826 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:52.682132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2039d14b-117b-4c02-bcaf-3bf1b7e3fd92 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.183624 Jul 31 01:10:52.682755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 126/258] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:52 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:52.823417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:52.824426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:52.837354 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:52.839120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:52.839916 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:52.841468 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:53.005737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:53.019197 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d9a3c77d-626d-45cb-b622-56bae5c8fa02 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.203092 Jul 31 01:10:53.021726 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 133/259] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:52 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:53.481229 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:53.485256 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:53.509797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:53.510595 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:53.518027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:53.519008 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:53.708840 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:53.709710 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:53.717681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:53.718417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:53.719590 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:53.720194 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:53.978378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:53.997863 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2a2f4099-fd01-495a-8b9c-2493b1a07957 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.526802 Jul 31 01:10:53.999003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 127/260] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:53 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:54.050855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:54.052073 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:54.067327 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:54.068698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:54.070456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:54.071492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:54.162926 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:54.180131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4ce43e7c-3e23-45d1-809b-313a5f97bd5e tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.477575 Jul 31 01:10:54.181825 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 134/261] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:53 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:54.269968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:54.286037 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-972d06ff-ec78-4b68-9e28-a9559cc61db2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.242540 Jul 31 01:10:54.286948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 128/262] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:54 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:55.035694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:55.036435 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:55.038628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=4 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:55.040305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=14 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:10:55.047777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:55.048364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:55.049257 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:55.049777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:55.214301 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:55.215102 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:55.223639 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:55.224272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:55.225257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:55.225874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:55.228848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:55.238975 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f6b609a3-ffb9-4bc7-b000-0ab709c0f7e9 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.207647 Jul 31 01:10:55.239979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 135/263] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:55 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:55.312126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:55.312729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:55.320403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:55.321251 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:55.322280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:55.322740 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:55.405619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:55.419269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-93969273-2119-487d-a1b7-aca8545b778f tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.207945 Jul 31 01:10:55.420300 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 129/264] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:55 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:55.506774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:55.517606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6a81b1d6-54f6-4c06-b526-cf7bfee22703 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.210082 Jul 31 01:10:55.518228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 136/265] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:55 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:56.264100 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:56.265372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:56.272565 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:56.273243 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:56.274046 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:56.274527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:56.441998 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:56.443174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:56.453223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:56.453903 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:56.454956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:56.455888 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:56.505503 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:56.516875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1c99cc86-3ac0-4de3-8de9-49cd0825d964 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.257830 Jul 31 01:10:56.532055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 130/266] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:56 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:56.549238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:56.549238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:56.565494 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:56.566380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:56.567642 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:56.568410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:56.684483 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:56.693202 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6eaa6dbb-d9c1-4f5a-b876-340880c20ce0 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.256390 Jul 31 01:10:56.694015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 137/267] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:56 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:56.770947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:56.780248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-704441c8-6e83-4bba-8f89-793e26b7c69d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.243387 Jul 31 01:10:56.780839 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 131/268] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:56 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:57.554064 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:57.556389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:57.570133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:57.571003 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:57.572474 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:57.573480 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:57.737816 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:57.739190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:57.751619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:57.752449 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:57.754400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:57.755737 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:57.943168 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:57.949488 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:57.955869 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1066be10-5bc3-4c9a-966f-8b66a194d12a tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.410209 Jul 31 01:10:57.957255 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 138/269] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:57 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:57.967736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:57.969312 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:57.986510 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b6caa386-54f5-4f7e-a6b7-94d0680f29d2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.263186 Jul 31 01:10:57.987831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:57.988682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:57.989866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 132/270] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:57 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:57.991696 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:57.992734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:58.354400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:58.365520 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d9b180b8-ea37-407e-a85e-801df41edfd6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.405636 Jul 31 01:10:58.366210 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 139/271] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:57 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:58.551671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-bc70a67f-d167-4e9f-8da8-fec9d7669581 req-958d11a7-6904-4f03-9f97-71715a67a396 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1fc0a86-e494-4707-bac6-ef80822d851b", "name": "network-vif-plugged", "status": "completed", "tag": "586fdcee-a89e-4e95-9f02-c485e653607f"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:10:58.572105 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-bc70a67f-d167-4e9f-8da8-fec9d7669581 req-958d11a7-6904-4f03-9f97-71715a67a396 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:58.573007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-bc70a67f-d167-4e9f-8da8-fec9d7669581 req-958d11a7-6904-4f03-9f97-71715a67a396 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:58.598579 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-bc70a67f-d167-4e9f-8da8-fec9d7669581 req-958d11a7-6904-4f03-9f97-71715a67a396 service nova] Creating event network-vif-plugged:586fdcee-a89e-4e95-9f02-c485e653607f for instance a1fc0a86-e494-4707-bac6-ef80822d851b on hv-cinder-maste Jul 31 01:10:58.611236 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-bc70a67f-d167-4e9f-8da8-fec9d7669581 req-958d11a7-6904-4f03-9f97-71715a67a396 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067821 Jul 31 01:10:58.612366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 133/272] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:10:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:10:59.006894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:59.007768 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:59.021621 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:59.023244 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:59.036384 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:59.037731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:59.040204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:59.040875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:59.041850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:59.042286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:59.059669 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:59.060725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:59.264672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:59.282494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-59b82d18-47d1-4606-a1a4-616d55e0e537 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.282837 Jul 31 01:10:59.284095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 140/273] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:58 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:10:59.328669 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:59.337366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d37b8a75-1ee8-4f8d-bae1-c2c6e92a6e69 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.321142 Jul 31 01:10:59.338145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 134/274] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:59 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:10:59.389001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:10:59.389994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:10:59.396666 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:59.397115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:59.397901 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:10:59.398336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:10:59.586734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:10:59.595649 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e9b32838-f1f8-47fa-9f74-5a250a9ecb27 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.211786 Jul 31 01:10:59.596544 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 141/275] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:10:59 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:00.318470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:00.319549 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:00.332499 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:00.333386 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:00.334705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:00.335677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:00.363915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:00.364950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:00.376497 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:00.377324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:00.378667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:00.379431 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:00.591168 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:00.607119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-18a3ae11-57db-47be-8fa2-a9823725cae0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.293787 Jul 31 01:11:00.609361 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 135/276] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:00 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:00.632746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:00.633638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:00.634616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:00.642438 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:00.642963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:00.644292 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:00.644772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:00.648759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-54950dff-0d93-46bf-b28f-cce0f9fdc701 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1314 microversion: 2.1 time: 0.287887 Jul 31 01:11:00.657071 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 142/277] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:00 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1314 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:00.690680 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-e292c40d-9a0b-48e8-aafa-faa83233f3ee req-a76c7b44-af24-45e6-85cf-3e99915167df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5", "name": "network-vif-plugged", "status": "completed", "tag": "a472a1b9-6f91-4385-b80e-65adbe320d3f"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:00.703829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-e292c40d-9a0b-48e8-aafa-faa83233f3ee req-a76c7b44-af24-45e6-85cf-3e99915167df service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:00.704286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-e292c40d-9a0b-48e8-aafa-faa83233f3ee req-a76c7b44-af24-45e6-85cf-3e99915167df service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:00.719686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-e292c40d-9a0b-48e8-aafa-faa83233f3ee req-a76c7b44-af24-45e6-85cf-3e99915167df service nova] Creating event network-vif-plugged:a472a1b9-6f91-4385-b80e-65adbe320d3f for instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 on hv-cinder-maste Jul 31 01:11:00.727396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-e292c40d-9a0b-48e8-aafa-faa83233f3ee req-a76c7b44-af24-45e6-85cf-3e99915167df service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042460 Jul 31 01:11:00.728377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 143/278] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:11:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:11:00.969658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:00.979753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3b844793-7829-4237-a5db-5c27918bb395 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.364429 Jul 31 01:11:00.993521 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 136/279] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:00 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:01.637143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:01.638751 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:01.648281 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:01.648927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:01.650146 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:01.650877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:01.681257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:01.684883 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:01.696788 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:01.698131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:01.699849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:01.700877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:01.905805 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:01.911553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:01.921417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-14cafe88-9612-4af2-84d2-1702020d26d0 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1314 microversion: 2.1 time: 0.289627 Jul 31 01:11:01.922491 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 144/280] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:01 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1314 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:01.932487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:01.940986 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e8fa988e-8aa9-47b7-b508-92c966a28ea7 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1463 microversion: 2.1 time: 0.268824 Jul 31 01:11:01.941678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 137/281] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:01 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1463 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:01.991632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:01.992453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:01.994885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:02.003097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.003599 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:02.004401 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.004844 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:02.005029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:02.005832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:02.016783 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.017157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:02.018653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.019202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:02.035702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:11:02.131080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c39abbd1-f215-4f09-a61d-84911803da54 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "DELETE /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 204 len: 0 microversion: 2.1 time: 0.143038 Jul 31 01:11:02.131862 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 145/282] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:11:01 2022] DELETE /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:11:02.145528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:02.146711 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:02.159187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.159698 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:02.160730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.161178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:02.260966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:02.270065 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d464f59d-5f89-434f-9686-086cc4a083a3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.271691 Jul 31 01:11:02.271029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 138/283] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:01 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:02.344958 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:02.356154 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cf7bef40-733d-4a7a-9553-ea0051a11155 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1469 microversion: 2.1 time: 0.217876 Jul 31 01:11:02.367411 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 146/284] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:02 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1469 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:02.944715 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:02.945342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:02.952418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.952869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:02.953649 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:02.954090 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.096946 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:03.104769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1f66aebb-fa41-4ca9-8c9b-febf62f47943 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1498 microversion: 2.1 time: 0.164285 Jul 31 01:11:03.105363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 139/285] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:02 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1498 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:03.307346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:03.308885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:03.317238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.317686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.318508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.318946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.391308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:03.392352 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:03.399319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.400142 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.402350 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.403035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.528814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:03.537456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3aabdca6-2236-45d1-ad19-e220044078aa tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.236784 Jul 31 01:11:03.538032 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 147/286] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:03 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:03.590408 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:03.598657 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a04d065f-72f9-47a2-9eda-fa2f213f9f28 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1469 microversion: 2.1 time: 0.212662 Jul 31 01:11:03.599235 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 140/287] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:03 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1469 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:03.812855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:03.813570 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:03.819409 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.819798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.820578 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.821015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.853361 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:11:03.947007 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-362f246b-41e6-4800-bf32-c1783c638fdd tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "DELETE /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 204 len: 0 microversion: 2.1 time: 0.139091 Jul 31 01:11:03.947706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 148/288] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:11:03 2022] DELETE /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:11:03.956276 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:03.956889 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:03.964192 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.964681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:03.965467 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:03.965909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:04.122109 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:04.134308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e2609541-db1e-4bc5-a833-6d9cabc81d16 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1504 microversion: 2.1 time: 0.181962 Jul 31 01:11:04.134905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 141/289] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:03 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1504 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:04.559857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:04.560689 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:04.567155 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:04.567632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:04.568492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:04.568836 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:04.619486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:04.620108 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:04.627509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:04.627967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:04.628757 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:04.629193 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:04.735738 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:04.749790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-dcd524c0-0c06-434f-a630-a8146ec16a79 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.194301 Jul 31 01:11:04.751084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 149/290] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:04 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:04.791839 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:04.795654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:04.800616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b386407a-c22b-4145-b10c-5726b42bcbfa tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1469 microversion: 2.1 time: 0.185124 Jul 31 01:11:04.801233 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 142/291] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:04 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1469 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:05.156439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:05.157201 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:05.163835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:05.164404 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:05.165071 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:05.165506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:05.330848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:05.339238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-22392a6b-3b1d-4ba2-8252-d307bec1ac6c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1504 microversion: 2.1 time: 0.187340 Jul 31 01:11:05.339875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 150/292] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:05 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1504 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:05.773930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:05.774549 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:05.782155 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:05.782602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:05.783422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:05.783870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:05.822345 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:05.823133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:05.830036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:05.830495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:05.831306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:05.831788 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:05.940804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:05.950606 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-85fbe616-8fdc-4276-a4f0-ef4b2a394614 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.181057 Jul 31 01:11:05.951199 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 143/293] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:05 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:06.189511 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:06.197919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0053789e-e5c1-4aa0-90ef-4b8d0a378d25 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1469 microversion: 2.1 time: 0.379502 Jul 31 01:11:06.198507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 151/294] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:05 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1469 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:06.364720 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:06.365348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:06.372257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:06.372714 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:06.373500 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:06.373941 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:06.514213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:06.522270 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ba1d07b1-3ead-484b-bf90-834072f4fd59 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1504 microversion: 2.1 time: 0.161428 Jul 31 01:11:06.522868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 144/295] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:06 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1504 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:06.972252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:06.972993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:06.979596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:06.980038 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:06.981394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:06.981824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:07.201187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:07.210161 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2c3e9e1e-c617-4439-a073-9d3e89d00881 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.241687 Jul 31 01:11:07.210669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 152/296] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:06 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:07.220638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:07.221903 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:07.231579 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:07.232057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:07.232920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:07.233435 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:07.384369 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:07.392615 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ae0809d9-ec3b-4f61-8f37-b113d2b0b1df tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1469 microversion: 2.1 time: 0.176495 Jul 31 01:11:07.393318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 145/297] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:07 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1469 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:07.543646 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:07.544712 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:07.551921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:07.552560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:07.554293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:07.554988 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:07.703324 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-025d2db6-ece0-4943-8251-0f165a335b7e req-16109a0d-8f8f-4dd4-adcb-ccc4c1635d4c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1fc0a86-e494-4707-bac6-ef80822d851b", "name": "network-vif-deleted", "tag": "586fdcee-a89e-4e95-9f02-c485e653607f"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:07.715585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-025d2db6-ece0-4943-8251-0f165a335b7e req-16109a0d-8f8f-4dd4-adcb-ccc4c1635d4c service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:07.716413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-025d2db6-ece0-4943-8251-0f165a335b7e req-16109a0d-8f8f-4dd4-adcb-ccc4c1635d4c service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:07.731783 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-025d2db6-ece0-4943-8251-0f165a335b7e req-16109a0d-8f8f-4dd4-adcb-ccc4c1635d4c service nova] Creating event network-vif-deleted:586fdcee-a89e-4e95-9f02-c485e653607f for instance a1fc0a86-e494-4707-bac6-ef80822d851b on hv-cinder-maste Jul 31 01:11:07.738434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-025d2db6-ece0-4943-8251-0f165a335b7e req-16109a0d-8f8f-4dd4-adcb-ccc4c1635d4c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039339 Jul 31 01:11:07.739746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 146/298] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:11:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:11:07.833405 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:07.844949 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4e677574-e1c6-4f2a-9f5d-e609140d33ff tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1504 microversion: 2.1 time: 0.305231 Jul 31 01:11:07.845997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 153/299] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:07 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1504 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:08.238745 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:08.240407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:08.247201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.247686 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.248474 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.248912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.417621 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:08.419215 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:08.426863 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.427584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.428548 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.429343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.466094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.466554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.467019 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lazy-loading 'fault' on Instance uuid a1fc0a86-e494-4707-bac6-ef80822d851b {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:11:08.480994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:08.489889 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-67845616-f8aa-45fc-9cfa-08eb67ed09cf tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.259205 Jul 31 01:11:08.493966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 147/300] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:08 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:08.497528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-ec746261-605f-47c7-ac2e-e9a945e88b3c req-ad3ad1a5-f9c2-4096-afe9-2be0859ba086 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5", "name": "network-vif-deleted", "tag": "a472a1b9-6f91-4385-b80e-65adbe320d3f"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:08.512827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ec746261-605f-47c7-ac2e-e9a945e88b3c req-ad3ad1a5-f9c2-4096-afe9-2be0859ba086 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.513619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ec746261-605f-47c7-ac2e-e9a945e88b3c req-ad3ad1a5-f9c2-4096-afe9-2be0859ba086 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.530653 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2e69ff835a0247869749c889143d8275', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bf5ca398012441cba969788634f537b8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:08.535103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-ec746261-605f-47c7-ac2e-e9a945e88b3c req-ad3ad1a5-f9c2-4096-afe9-2be0859ba086 service nova] Creating event network-vif-deleted:a472a1b9-6f91-4385-b80e-65adbe320d3f for instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 on hv-cinder-maste Jul 31 01:11:08.539995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2bc43ab7-a6da-4545-91ef-a3556b7b24b3 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 200 len: 1304 microversion: 2.1 time: 0.129129 Jul 31 01:11:08.540845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 154/301] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:08 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 1304 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:08.544073 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-ec746261-605f-47c7-ac2e-e9a945e88b3c req-ad3ad1a5-f9c2-4096-afe9-2be0859ba086 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051504 Jul 31 01:11:08.545182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 148/302] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:11:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:11:08.869641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:08.870174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:08.879963 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.881552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.881882 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:08.882519 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:08.974157 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8fcc34c42bb64200ac5f40d770d6784b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de83743a559b4a289aa40c8d16641ee9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:08.986578 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-02992575-064a-40ae-a1ea-28d9fb17dc18 tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 200 len: 1299 microversion: 2.1 time: 0.121591 Jul 31 01:11:08.988038 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 155/303] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:08 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 1299 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:09.520375 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:09.521131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:09.523571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=84 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:09.529990 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:09.531124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:09.532467 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:09.533201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:09.565088 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:09.565866 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] [instance: a1fc0a86-e494-4707-bac6-ef80822d851b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:09.572934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:09.580484 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:09.581392 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:09.581850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:09.703997 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:09.712714 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3cb712b2-a640-42be-a276-e429d5a0acd8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.197052 Jul 31 01:11:09.713413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 149/304] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:09 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:09.755782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] HTTP exception thrown: Instance a1fc0a86-e494-4707-bac6-ef80822d851b could not be found. Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Request method failure captured: Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b HTTP/1.1 Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5daOVfx0mUZtvQQnD8XZKI1Zh-N8n1apH-XAW5G6MmDIZ56Ik_of5qhjghhTTWDlIeayd88w2vWWnBncOIJM8aSjvMdTldQYmkjTuoG3t2PS2-L_J8qA6Yt-BCc238FYMsn_E0w5z2I1rCO0AYfeQtx5O8sdC5hHmRQR0GZTuTlAjaY Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: bf5ca398012441cba969788634f537b8 Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesNegativeTest-1698942776 Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:11:09.757454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/bf5ca398012441cba969788634f537b8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bf5ca398012441cba969788634f537b8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bf5ca398012441cba969788634f537b8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_bf5ca398012441cba969788634f537b8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesNegativeTest-1698942776 Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: bf5ca398012441cba969788634f537b8 Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesNegativeTest-1698942776 Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesNegativeTest-1698942776-project Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 2e69ff835a0247869749c889143d8275 Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesNegativeTest-1698942776-project Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance a1fc0a86-e494-4707-bac6-ef80822d851b could not be found. Jul 31 01:11:09.760504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'a1fc0a86-e494-4707-bac6-ef80822d851b'} Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1fc0a86-e494-4707-bac6-ef80822d851b could not be found. Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:11:09.762628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1fc0a86-e494-4707-bac6-ef80822d851b could not be found. Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1fc0a86-e494-4707-bac6-ef80822d851b could not be found. Jul 31 01:11:09.764794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:11:09.766430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] Returning 404 to user: Instance a1fc0a86-e494-4707-bac6-ef80822d851b could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:11:09.766430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7df54fe4-fe70-424e-8b1b-c65b88773fb2 tempest-VolumesNegativeTest-1698942776 tempest-VolumesNegativeTest-1698942776-project] 10.222.0.42 "GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b" status: 404 len: 111 microversion: 2.1 time: 0.198669 Jul 31 01:11:09.766430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 156/305] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:09 2022] GET /compute/v2.1/servers/a1fc0a86-e494-4707-bac6-ef80822d851b => generated 111 bytes in 200 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:11:10.008173 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:10.008790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] [instance: 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:10.015398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:10.015849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:10.016666 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:10.017098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:10.038679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] HTTP exception thrown: Instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 could not be found. Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Request method failure captured: Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 HTTP/1.1 Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dZrdLW10-XzTesHhUghgJp-iRJC_pyNwHJoAzstK9NY9Kjc5nxm7NUbHv2P4U_LQfq5nI2id6MYONtowHj2QUy2x6xH64Wy1DJo1GNcXJi4v4oxetOAgJex_7HVeLOx6PN52FV1PLWOi9fD8G-3pXvreD-F4kelhe_elibIreX9fGk Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: de83743a559b4a289aa40c8d16641ee9 Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-VolumesActionsTest-1592492359 Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:11:10.040395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/de83743a559b4a289aa40c8d16641ee9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/de83743a559b4a289aa40c8d16641ee9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/de83743a559b4a289aa40c8d16641ee9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_de83743a559b4a289aa40c8d16641ee9", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-VolumesActionsTest-1592492359 Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: de83743a559b4a289aa40c8d16641ee9 Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-VolumesActionsTest-1592492359 Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-VolumesActionsTest-1592492359-project Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 8fcc34c42bb64200ac5f40d770d6784b Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-VolumesActionsTest-1592492359-project Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 could not be found. Jul 31 01:11:10.041761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5'} Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 could not be found. Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:11:10.043112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 could not be found. Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 could not be found. Jul 31 01:11:10.044971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:11:10.047905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] Returning 404 to user: Instance 39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:11:10.047905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2da46eca-f668-4435-aa5b-f9a038dd599c tempest-VolumesActionsTest-1592492359 tempest-VolumesActionsTest-1592492359-project] 10.222.0.42 "GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5" status: 404 len: 111 microversion: 2.1 time: 0.037802 Jul 31 01:11:10.047905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 150/306] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:10 2022] GET /compute/v2.1/servers/39a7d8b1-bf40-4a9d-aa85-d6ef014c6ee5 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:11:10.735232 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:10.735725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:10.745381 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:10.745864 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:10.746636 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:10.747067 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:10.924270 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:10.936044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-69eaad01-1054-4968-94ba-60a638aa52e1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.205045 Jul 31 01:11:10.936622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 157/307] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:10 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:11.971136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:11.972439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:11.981313 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:11.981769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:11.982815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:11.983131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:12.144275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:12.155806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4e41afdc-3175-479c-8358-09235dcc7469 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.192538 Jul 31 01:11:12.157288 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 151/308] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:11 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:13.178894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:13.179542 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:13.186915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:13.187380 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:13.188332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:13.188838 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:13.209426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:13.330573 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:13.338467 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0bd45e98-fec7-45c2-8350-02e893150308 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.163389 Jul 31 01:11:13.338961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 158/309] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:13 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:14.361316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:14.361947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:14.370372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:14.371471 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:14.372262 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:14.372699 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:14.512742 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:14.525356 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9796e2a8-4c73-4c1e-8c9b-be95486e9180 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.168123 Jul 31 01:11:14.526553 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 152/310] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:14 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:15.556538 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:15.557277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:15.564582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:15.565037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:15.565819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:15.566259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:15.703721 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:15.713648 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-192623f7-0e3e-4e88-941f-3e7e2bf93475 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.162413 Jul 31 01:11:15.714851 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 159/311] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:15 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:16.734533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:16.735203 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:16.742282 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:16.742727 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:16.743576 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:16.744010 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:16.891884 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:16.900067 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-509a5f4c-12ef-4232-9f5b-9104fe7c9751 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.169402 Jul 31 01:11:16.900673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 153/312] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:16 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:17.921406 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:17.921997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:17.928754 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:17.929130 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:17.929907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:17.930342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:18.092316 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:18.100212 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b446e515-6231-4c9e-a159-b4be9233d672 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.183150 Jul 31 01:11:18.100804 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 160/313] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:17 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:19.124754 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:19.125623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:19.133937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:19.134396 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:19.135193 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:19.135863 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:19.308872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:19.323179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2556bf9e-1470-4a57-ba78-38bccd971c5d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.203803 Jul 31 01:11:19.324206 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 154/314] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:19 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:19.579021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-78b184bc-9be2-46f3-93c1-9d10c5c3fdef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:19.580453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-78b184bc-9be2-46f3-93c1-9d10c5c3fdef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:19.592154 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78b184bc-9be2-46f3-93c1-9d10c5c3fdef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:19.593412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78b184bc-9be2-46f3-93c1-9d10c5c3fdef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:19.594758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78b184bc-9be2-46f3-93c1-9d10c5c3fdef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:19.595646 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78b184bc-9be2-46f3-93c1-9d10c5c3fdef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:19.631037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-78b184bc-9be2-46f3-93c1-9d10c5c3fdef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/ips" status: 200 len: 150 microversion: 2.1 time: 0.059390 Jul 31 01:11:19.644350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 161/315] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:11:19 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/ips => generated 150 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:11:20.358657 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:20.359879 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:20.371966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:20.376202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:20.379671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:20.380381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:20.651327 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:20.667768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8a6888b3-24e6-4ff1-8501-260fccbbb52a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.312714 Jul 31 01:11:20.669444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 155/316] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:20 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:21.398024 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f9e34dbd-0cdd-4e26-a2b4-df6e660e2806 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "44ee31f1-a5f4-4b51-b90c-15f7f7430a97"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:21.399926 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f9e34dbd-0cdd-4e26-a2b4-df6e660e2806 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:21.406344 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9e34dbd-0cdd-4e26-a2b4-df6e660e2806 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:21.406797 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9e34dbd-0cdd-4e26-a2b4-df6e660e2806 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:21.407601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9e34dbd-0cdd-4e26-a2b4-df6e660e2806 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:21.408040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9e34dbd-0cdd-4e26-a2b4-df6e660e2806 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:21.714611 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:21.717485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:21.736319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:21.738340 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:21.740796 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:21.742371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:21.903814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f9e34dbd-0cdd-4e26-a2b4-df6e660e2806 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.509573 Jul 31 01:11:21.905077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 162/317] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:11:21 2022] POST /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments => generated 194 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:11:21.988809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:22.000472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3eafc2ca-a853-4541-bef6-5336e85dcd45 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.300781 Jul 31 01:11:22.001073 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 156/318] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:21 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:23.024810 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:23.025507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:23.032555 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:23.033010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:23.033800 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:23.034249 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:23.199538 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:23.207839 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4b1570ce-dbe0-4e22-9e16-bafcb48ad5f7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.187643 Jul 31 01:11:23.208443 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 163/319] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:23 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:24.235019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:24.245214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:24.275691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:24.275691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:24.275691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:24.275691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:24.475867 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:24.486151 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9c5b04fc-2fbe-4dc9-b39a-e62668869ed0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.260747 Jul 31 01:11:24.487022 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 157/320] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:24 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:25.516392 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:25.517406 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:25.539451 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:25.540488 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:25.541550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:25.542192 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:25.726910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:25.734946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c00dfa05-89a3-472c-9144-ac1b9df86681 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.226411 Jul 31 01:11:25.735560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 164/321] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:25 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:26.761124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:26.762299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:26.773522 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:26.774494 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:26.776078 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:26.777718 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:26.955238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:26.967194 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-54105c3e-c162-4c70-9ae9-804d4b86ee6f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.213734 Jul 31 01:11:26.968171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 158/322] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:26 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:28.007475 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:28.009993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:28.021832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:28.022540 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:28.023544 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:28.024498 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:28.259597 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:28.275705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-efd42a9e-7c61-4202-896c-84283eb6a765 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.275546 Jul 31 01:11:28.277483 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 165/323] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:27 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:29.311415 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:29.312308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:29.320213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:29.321355 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:29.322134 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:29.322750 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:29.508104 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:29.532196 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-08e969c9-e40f-4ae3-a693-16ac4d86cc55 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.225870 Jul 31 01:11:29.533254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 159/324] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:29 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:30.589595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:30.590724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:30.600549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:30.601361 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:30.602809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:30.603390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:30.802627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:30.818173 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c969ba12-f68d-4e5e-8481-01751f7ac322 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.253941 Jul 31 01:11:30.819178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 166/325] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:30 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:31.360191 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3e7ab137-4ada-412e-90eb-f246c349720c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:31.361359 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3e7ab137-4ada-412e-90eb-f246c349720c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:31.371151 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3e7ab137-4ada-412e-90eb-f246c349720c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:31.371777 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3e7ab137-4ada-412e-90eb-f246c349720c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:31.372697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3e7ab137-4ada-412e-90eb-f246c349720c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:31.373140 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3e7ab137-4ada-412e-90eb-f246c349720c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:31.416429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3e7ab137-4ada-412e-90eb-f246c349720c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/ips" status: 200 len: 152 microversion: 2.20 time: 0.062458 Jul 31 01:11:31.424677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 160/326] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Jul 31 01:11:31 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/ips => generated 152 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:11:31.860509 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:31.861462 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:31.872658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:31.873682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:31.875384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:31.876279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:32.178934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:32.192057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c3a896b3-3c52-440d-8e1c-abd170dccfd1 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.340328 Jul 31 01:11:32.193679 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 167/327] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:31 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:33.216245 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:33.217618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:33.225425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:33.225974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:33.227199 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:33.227835 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:33.807284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:33.814863 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:33.821657 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7cc5c1bb-d1a1-4ca6-8574-183381f3c5ae tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.609776 Jul 31 01:11:33.840047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 161/328] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:33 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:34.315477 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-435472b5-b013-4d61-9d1e-9edff058ab17 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: .version_select at 0x7f3e801f81f0>, body: {"keypair": {"name": "tempest-keypair-1892841760", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDagFFEmPSLODMG12lS8UwbHS9u6ZRrqt1fG3LfhzWm8gF5xKr15Tzp9BjHoVhMVAZYd+mdsXXzsU6g/MiRH59D6YtfwsFaDDh+jp7sqOTnx2P4r31flncdN3LkFhAU0vA=="}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:34.330128 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:34.348319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-435472b5-b013-4d61-9d1e-9edff058ab17 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:11:34.356912 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-435472b5-b013-4d61-9d1e-9edff058ab17 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:11:34.392832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR oslo.messaging._drivers.impl_rabbit [None req-435472b5-b013-4d61-9d1e-9edff058ab17 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [fc076a8a-d6fb-4b06-b093-3666bb855a1b] AMQP server on 10.222.0.42:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 31 01:11:34.865926 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:34.869695 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:34.879937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:34.879937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:34.880990 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:34.888871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:35.131772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:35.159574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-443daac4-63d5-4f50-9f9e-b123ef15e120 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.311164 Jul 31 01:11:35.168478 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 162/329] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:34 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:35.186928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:35.190326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:35.207115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:35.208229 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:35.212137 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:35.212982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:35.273566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lazy-loading 'flavor' on Instance uuid ef19c2df-372c-4c77-ad6c-ada1cd419a14 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:11:35.411276 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO oslo.messaging._drivers.impl_rabbit [None req-435472b5-b013-4d61-9d1e-9edff058ab17 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [fc076a8a-d6fb-4b06-b093-3666bb855a1b] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 36124. Jul 31 01:11:35.453609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-435472b5-b013-4d61-9d1e-9edff058ab17 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.143818 Jul 31 01:11:35.468233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 168/330] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:11:34 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:11:35.596300 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d786cebf-4e92-4bef-b679-8781e54f7056 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/action" status: 202 len: 0 microversion: 2.20 time: 0.426570 Jul 31 01:11:35.601077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 163/331] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Jul 31 01:11:35 2022] POST /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/action => generated 0 bytes in 433 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:11:35.614294 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:35.617446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:35.637858 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:35.638601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:35.640345 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:35.641518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:35.969506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:35.983584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:35.985848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:35.987387 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-550022e0-524b-4b83-83dd-ae14746c8e20 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.384787 Jul 31 01:11:36.003193 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 169/332] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:35 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:36.194231 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:36.195635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:36.206559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:36.207626 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:36.208683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:36.209297 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:36.676868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:36.688181 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d96b3015-4c17-4588-ab9d-12cb552407ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.498129 Jul 31 01:11:36.703842 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 164/333] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:36 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:37.015697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:37.017844 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:37.031385 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:37.033734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:37.035530 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:37.036581 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:37.312894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:37.330103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:37.336072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:37.340381 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-21c9a250-f939-4c40-b6e1-a5983941f702 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.331954 Jul 31 01:11:37.344235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 170/334] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:37 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:37.723087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:37.724237 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:37.738706 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:37.739932 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:37.741683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:37.742519 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:38.217091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:38.235509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a19020be-a9ce-433d-aec6-f94ecdcf862c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.520831 Jul 31 01:11:38.237013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 165/335] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:37 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:38.411087 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:38.415709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:38.425836 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:38.427024 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:38.430075 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:38.431421 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:38.632704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:38.662364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:38.662364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:38.662364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-db3a9cb8-7f59-4d4c-b37a-090365e8897a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.257586 Jul 31 01:11:38.663492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 171/336] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:38 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:38.761158 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2112059293", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b"}], "security_groups": [{"name": "tempest-securitygroup--149165326"}], "key_name": "tempest-keypair-1892841760", "user_data": "}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:38.996094 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] validate_networks() for [('3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:11:39.239787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:11:39.240621 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:11:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.42) !!! Jul 31 01:11:39.263193 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:39.263850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:39.272270 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:39.273012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:39.274105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:39.274527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:39.441661 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:39.472856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-24f5cb5a-5210-462a-ac45-88010f1694cd tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.204041 Jul 31 01:11:39.474029 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 172/337] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:39 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:39.706140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:39.707309 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:39.718786 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:39.719639 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:39.721110 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:39.721905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:39.733524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:11:39.734195 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:11:39.734804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:11:39.735813 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:11:39.736425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:11:39.738900 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=38 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:39.756079 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:39.756820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:39.758165 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:39.758906 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:39.784485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:11:39.792468 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:11:39.809118 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:11:39.812518 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:11:39.886382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f755930e-b698-44d8-ab4b-5c8ef12775b9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.132257 Jul 31 01:11:39.887990 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 166/338] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:11:38 2022] POST /compute/v2.1/servers => generated 401 bytes in 1136 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:11:39.894298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:39.907004 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d50e165e-0acb-4d85-a4fb-876097f7a7a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:39.908557 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d50e165e-0acb-4d85-a4fb-876097f7a7a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:39.910034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:39.916961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:39.919295 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-78c833f3-c85f-4434-b45e-ca2493c488f0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.221727 Jul 31 01:11:39.920742 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 173/339] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:39 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:11:39.991962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d50e165e-0acb-4d85-a4fb-876097f7a7a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:40.011286 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d50e165e-0acb-4d85-a4fb-876097f7a7a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1242 microversion: 2.1 time: 0.110898 Jul 31 01:11:40.012477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 167/340] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:39 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1242 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:40.502729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:40.503428 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:40.510882 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:40.511489 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:40.512512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:40.512966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:40.690541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:40.701465 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6a5c9267-5a0e-4110-ad65-f2110801cd25 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.202850 Jul 31 01:11:40.702237 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 174/341] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:40 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:40.954966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:40.956753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:40.966633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:40.967420 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:40.968644 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:40.969157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:41.051420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:41.052906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:41.066362 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:41.067223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:41.068760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:41.069669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:41.079444 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:41.193467 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:41.194425 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:41.204250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:41.207622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:41.207862 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-02de85e8-7a0d-4b8b-b006-c9ba41da5d56 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1238 microversion: 2.1 time: 0.163185 Jul 31 01:11:41.208123 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-eb3438fb-0467-48ab-aa2f-d9daa265ffa4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.261136 Jul 31 01:11:41.208867 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 168/342] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:40 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:41.209008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 175/342] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:41 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1238 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:41.734974 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:41.736241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:41.761437 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:41.762275 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:41.769277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:41.770532 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:41.986687 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:41.997565 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-23f1ef40-8f20-4438-8563-f43b8e070194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.273217 Jul 31 01:11:41.998588 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 176/344] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:41 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:42.234033 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:42.234959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:42.241555 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:42.244049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:42.244049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:42.244049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:42.252522 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:42.253518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:42.265410 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:42.266145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:42.267197 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:42.267885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:42.361076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:42.375951 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b7ebae92-3b7b-4ac6-8bc5-778f5b531e6c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1300 microversion: 2.1 time: 0.146301 Jul 31 01:11:42.376940 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 169/345] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:42 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1300 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:42.465233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:42.480461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:42.483644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:42.485865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e48d7f9b-afbe-4c9e-8bfc-e3352275d3a9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.238911 Jul 31 01:11:42.486841 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 177/346] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:42 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:43.038552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:43.039337 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:43.047200 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:43.047798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:43.048687 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:43.049217 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:43.258290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:43.267215 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8933db10-16eb-4d6f-ac9a-53a5911ee459 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.232856 Jul 31 01:11:43.267933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 170/347] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:43 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:43.406028 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:43.407064 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:43.417920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:43.418733 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:43.420279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:43.421109 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:43.542546 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:43.544098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:43.557834 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:43.559234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:43.560557 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:43.561779 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:43.666930 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:43.680085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-068dd3fc-2c59-45be-b4eb-4f287223bebd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.278633 Jul 31 01:11:43.680911 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 178/348] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:43 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:44.008451 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:44.019762 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:44.021628 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:44.023645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9b082409-b6ca-4559-b7f7-40db7a89f6ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.503835 Jul 31 01:11:44.024754 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 171/349] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:43 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:44.296150 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:44.297443 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:44.311566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:44.312384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:44.313778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:44.314530 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:44.368622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=4 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:44.515875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:44.528394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-92288500-d339-46e0-a132-13fcc7e8ea54 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.240906 Jul 31 01:11:44.529267 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 179/350] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:44 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:44.647070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-b6edb5d7-ad24-485e-88cc-9b7a1c783b0d req-b7b74f8e-8aed-4581-9029-2befa8a71e4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2b53247d-da0a-46f9-aa9e-f4d09342e5bf", "tag": "b48b5519-311b-4ddf-9138-fa82f7ac77af"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:44.658897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b6edb5d7-ad24-485e-88cc-9b7a1c783b0d req-b7b74f8e-8aed-4581-9029-2befa8a71e4e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:44.659376 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b6edb5d7-ad24-485e-88cc-9b7a1c783b0d req-b7b74f8e-8aed-4581-9029-2befa8a71e4e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:44.673032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-b6edb5d7-ad24-485e-88cc-9b7a1c783b0d req-b7b74f8e-8aed-4581-9029-2befa8a71e4e service nova] Creating event network-changed:b48b5519-311b-4ddf-9138-fa82f7ac77af for instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf on hv-cinder-maste Jul 31 01:11:44.678703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-b6edb5d7-ad24-485e-88cc-9b7a1c783b0d req-b7b74f8e-8aed-4581-9029-2befa8a71e4e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036338 Jul 31 01:11:44.679667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 172/351] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:11:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:11:44.707444 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:44.708070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:44.714782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:44.715226 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:44.716049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:44.716489 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:44.884337 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:44.892831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-619f1024-4c1f-410b-9afa-1b111f49b95c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.189550 Jul 31 01:11:44.893545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 180/352] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:44 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:45.049348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:45.050106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:45.058024 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:45.058480 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:45.059277 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:45.059764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:45.228299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:45.243863 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:45.246931 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:45.249082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-122b6d4f-9823-46a1-9db7-ae79a4cb1e10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.204238 Jul 31 01:11:45.250177 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 173/353] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:45 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:11:45.560355 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:45.561025 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:45.568667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:45.569179 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:45.575273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:45.575273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:45.765506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:45.773577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d057b515-2024-41e3-97f2-ffdd3f97a059 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.217427 Jul 31 01:11:45.774189 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 181/354] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:45 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:45.920177 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:45.921369 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:45.934025 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:45.934580 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:45.935473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:45.935964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:46.101212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:46.110618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-801dedbb-7000-4060-a59a-8c05cd41da95 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.197623 Jul 31 01:11:46.111228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 174/355] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:45 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:46.287879 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:46.288677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:46.297257 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:46.297708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:46.298500 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:46.298936 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:46.471896 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:46.484034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:46.487087 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:46.489324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5caa4a50-6ee5-49c2-bf5a-9bc5a70e02c9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.205850 Jul 31 01:11:46.490302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 182/356] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:46 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:46.798940 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:46.799601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:46.806468 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:46.806926 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:46.808069 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:46.808506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:46.989937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:46.999016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-39666e66-2d11-47ea-92a5-ba048c718fe8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1523 microversion: 2.1 time: 0.204166 Jul 31 01:11:46.999774 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 175/357] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:46 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1523 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:47.133632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:47.134260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:47.141846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:47.142325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:47.143151 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:47.143676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:47.324763 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:47.333551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1cc20a9d-9904-49ad-a1c8-ef894e030bc9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.204144 Jul 31 01:11:47.334286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 183/358] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:47 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:47.514059 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:47.514710 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:47.521875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:47.522384 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:47.523276 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:47.523763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:47.685720 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:47.695984 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:47.698244 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:47.699602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-962cb0d9-b0b1-4d90-b1d1-e5679b36f9b1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.189899 Jul 31 01:11:47.700272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 176/359] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:47 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:11:48.021987 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:48.022600 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:48.029105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.029558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:48.030342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.030778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:48.223495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:48.234100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-300e3c18-82ca-4d9f-92c5-0ac85f22addc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1503 microversion: 2.1 time: 0.216256 Jul 31 01:11:48.235111 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 184/360] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:48 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1503 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:48.263679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:48.264558 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:48.284181 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.284753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:48.285675 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.290383 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:48.360753 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:48.363460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:48.372623 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.373380 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:48.374519 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.375204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:48.644631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:48.661118 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5115a534-7564-43ad-a197-ae0953d3ef07 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.303959 Jul 31 01:11:48.662968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 185/361] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:48 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:48.731475 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:48.733145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:48.760656 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.761443 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:48.762994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:48.772301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:49.007017 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:49.017740 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f203b7d0-11d5-412f-ab3a-1a14f7acc73a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1517 microversion: 2.1 time: 0.759347 Jul 31 01:11:49.018353 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 177/362] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:48 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1517 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:49.359680 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:49.373937 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:49.375657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:49.377677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f8a1c638-74ca-42e6-9616-4037f914161a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.654498 Jul 31 01:11:49.378501 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 186/363] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:48 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:11:49.703582 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:49.704202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:49.723092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:49.723588 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:49.724398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:49.730009 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:49.779259 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:50.001482 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=8 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:50.027705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:50.031757 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:50.039170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-057c84b5-22eb-4942-b3c1-ee726eb80bb6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.355077 Jul 31 01:11:50.040492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 178/364] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:49 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:50.417219 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:50.418127 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:50.436605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:50.437421 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:50.438845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:50.439642 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:50.731004 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:50.743675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:50.746050 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:50.747317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e16f92af-9814-4114-9fdd-c3be8c3fb263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.334837 Jul 31 01:11:50.764086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 187/365] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:50 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:11:50.966749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:11:50.973831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:50.989816 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:50.990979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:50.992436 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:50.993191 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:51.028697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lazy-loading 'flavor' on Instance uuid d2206492-6f7a-4a00-ac23-63200f66c940 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:11:51.072078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:51.073547 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:51.088135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:51.089149 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:51.090954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:51.091979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:51.219596 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-15bb8af4-4f59-49c2-a77c-39324a78c2ab tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940/action" status: 200 len: 32 microversion: 2.1 time: 0.259101 Jul 31 01:11:51.224969 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 179/366] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:11:50 2022] POST /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940/action => generated 32 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:11:51.245073 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:51.246560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:51.263797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:51.265641 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:51.269075 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:51.270152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:51.398099 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:51.410000 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-abd452a8-7ccf-4625-b992-985a7540f0ff tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.351291 Jul 31 01:11:51.411733 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 188/367] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:51 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:51.558382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:51.567279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-28ea3d45-6462-487f-8ae4-a18d00a77cb5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.332433 Jul 31 01:11:51.567926 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 180/368] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:51 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:51.776371 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:51.777571 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:51.789277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:51.790067 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:51.792160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:51.792934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:52.005238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:52.016159 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:52.017925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:52.019499 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-559614ce-697f-4232-bd42-a51f10f3a263 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.250475 Jul 31 01:11:52.020396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 189/369] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:51 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:52.448965 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:52.450392 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:52.460379 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:52.461200 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:52.463034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:52.463962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:52.596955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:52.599196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:52.613601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:52.614968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:52.616791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:52.617737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:52.692618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:52.703082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8f0e0b15-aebb-41bb-bb47-7b30cecfb62d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.261121 Jul 31 01:11:52.704113 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 181/370] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:52 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:52.828733 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:52.837101 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-401e55d2-16ba-4542-9d4a-4c7173ce4c5b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.250640 Jul 31 01:11:52.838302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 190/371] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:52 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:53.046151 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:53.046668 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:53.054281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:53.054916 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:53.055864 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:53.056318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:53.200056 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:53.208860 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:53.210396 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:53.211639 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0312a55b-eeb8-47b6-9c65-38409d11f92f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.169653 Jul 31 01:11:53.212221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 182/372] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:53 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:53.726200 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:53.726827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:53.733884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:53.734331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:53.735122 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:53.735600 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:53.858931 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:53.860343 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:53.867945 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:53.868560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:53.869979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:53.870629 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:53.890188 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:53.893984 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:11:53.899032 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8ab33d89-1e58-451e-bb4a-9833501de84c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.177274 Jul 31 01:11:53.899662 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 191/373] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:53 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:54.018203 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:54.028707 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-30712f51-a9ad-4f35-aaba-66d6c56a65bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.172781 Jul 31 01:11:54.028707 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 183/374] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:53 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:54.236514 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:54.237251 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:54.244746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:54.245252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:54.246080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:54.246558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:54.436384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:54.445885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:54.448084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:54.449656 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3f2a1bd2-d346-4745-b078-b5a0ac8d4875 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.218741 Jul 31 01:11:54.450284 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 192/375] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:54 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:54.921382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:54.922487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:54.929937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:54.930504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:54.932371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:54.932817 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:55.050946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:55.052057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:55.061905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:55.062505 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:55.063496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:55.064094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:55.104928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:55.119619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f792a5f4-4563-43ab-873b-ed211554c901 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.201411 Jul 31 01:11:55.120730 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 184/376] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:54 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:55.238243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:55.247470 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7bf76485-ae22-460a-aff4-5d9b8eaf2897 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.200817 Jul 31 01:11:55.248149 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 193/377] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:55 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:55.486069 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:55.486807 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:55.495106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:55.495610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:55.496418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:55.497106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:55.718787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:55.727679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:55.731347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:55.733680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0c11eccb-ebed-402a-9d42-189d8d8a607b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.252521 Jul 31 01:11:55.734740 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 185/378] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:55 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:56.147505 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:56.148300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:56.156133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:56.156826 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:56.157672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:56.158156 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:56.282634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:56.284960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:56.295767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:56.296568 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:56.297729 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:56.298520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:56.347574 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:56.357173 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3683b6df-141f-46ab-b1e1-2784c7d2e531 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.214686 Jul 31 01:11:56.357800 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 194/379] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:56 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:56.466964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:56.475828 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ceb5caae-866c-4fcd-bdc1-1b3b9dccdaa8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.198275 Jul 31 01:11:56.476466 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 186/380] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:56 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:11:56.774628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:56.775412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:56.783828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:56.784273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:56.785065 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:56.785499 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:56.985742 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:57.009814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:57.021462 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:57.024203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6a2d06cc-a942-432d-871a-5eb40b05c435 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.253381 Jul 31 01:11:57.027556 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 195/381] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:56 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:57.380993 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:57.383516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:57.409791 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:57.410572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:57.413043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:57.416345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:57.502737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:57.503933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:57.517376 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:57.518798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:57.520652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:57.521903 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:57.755828 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:57.778106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ab0ded98-df43-4796-93d2-3428b13d9379 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.403115 Jul 31 01:11:57.780962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 187/382] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:57 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:57.911178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:57.920062 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-11fbb0a6-5e90-4a44-bbfe-e4c3de8d246f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.425598 Jul 31 01:11:57.920669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 196/383] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:57 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:58.062574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:58.063587 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:58.071254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:58.075337 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:58.076815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:58.078806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:58.419150 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:58.432698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:58.434455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:58.435792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8c0c702a-e8aa-475f-ab9e-530f1d0dc985 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.380209 Jul 31 01:11:58.436511 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 188/384] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:58 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:11:58.825823 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:58.828276 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:58.846390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:58.848514 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:58.850197 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:58.852090 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:58.946443 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:58.948034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:58.959068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:58.959871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:58.961245 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:58.961966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:59.181827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:59.197414 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7e1a033a-25a6-4da5-afd0-353d3bbf1469 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.381153 Jul 31 01:11:59.198429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 197/385] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:58 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:59.244141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:59.256274 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4433e190-26f4-4912-afe4-38e46707b48e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.316425 Jul 31 01:11:59.257328 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 189/386] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:11:58 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:11:59.465450 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:11:59.466137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:11:59.473332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:59.473783 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:59.474571 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:11:59.475039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:11:59.636461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:59.648590 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:59.650677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:11:59.653002 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4780e82f-bf14-4e58-b7ee-63777af1a0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.192078 Jul 31 01:11:59.654060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 198/387] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:11:59 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:00.036987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-1bd53a5a-34b2-494d-8fd8-5063ee83e6ec req-170fdb7a-f7fe-470f-9146-a6231198cb1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b53247d-da0a-46f9-aa9e-f4d09342e5bf", "name": "network-vif-plugged", "status": "completed", "tag": "b48b5519-311b-4ddf-9138-fa82f7ac77af"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:00.048575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1bd53a5a-34b2-494d-8fd8-5063ee83e6ec req-170fdb7a-f7fe-470f-9146-a6231198cb1a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:00.049418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1bd53a5a-34b2-494d-8fd8-5063ee83e6ec req-170fdb7a-f7fe-470f-9146-a6231198cb1a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:00.063540 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-1bd53a5a-34b2-494d-8fd8-5063ee83e6ec req-170fdb7a-f7fe-470f-9146-a6231198cb1a service nova] Creating event network-vif-plugged:b48b5519-311b-4ddf-9138-fa82f7ac77af for instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf on hv-cinder-maste Jul 31 01:12:00.071051 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-1bd53a5a-34b2-494d-8fd8-5063ee83e6ec req-170fdb7a-f7fe-470f-9146-a6231198cb1a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037354 Jul 31 01:12:00.072024 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 190/388] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:12:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:12:00.236734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:00.238494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:00.262319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:00.263256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:00.264675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:00.265480 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:00.297439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:00.299292 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:00.311083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:00.312091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:00.313215 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:00.313765 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:00.566104 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:00.574528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:00.584269 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ea98426a-3255-45b6-9763-57ead39b9f8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.355925 Jul 31 01:12:00.586205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 199/389] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:00 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:00.611281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a2fe3a27-d915-44f8-861c-2589943fd60c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.320698 Jul 31 01:12:00.612330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 191/390] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:00 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:00.681524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:00.683070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:00.694403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:00.695208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:00.697386 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:00.698196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:00.896049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:00.912718 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:00.916047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:00.918236 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a836a42c-b272-497e-8b02-3f5a0e06b275 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.244591 Jul 31 01:12:00.919620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 200/391] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:00 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:01.616391 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:01.618168 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:01.626400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:01.626847 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:01.627659 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:01.628087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:01.639677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:01.639677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:01.645625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:01.646415 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:01.647676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:01.648351 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:01.846145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:01.855688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-14d7f02d-9fe3-4367-9b04-518c825a1cd6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1367 microversion: 2.1 time: 0.244835 Jul 31 01:12:01.856280 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 192/392] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:01 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1367 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:01.862303 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:01.870370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8979bcac-0047-4d25-aeb2-48c07f88945c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.240438 Jul 31 01:12:01.870946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 201/393] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:01 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:01.945786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:01.948118 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:01.961110 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:01.966983 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:01.968760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:01.969811 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:02.154284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:02.163552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:02.165104 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:02.166332 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d336a8c7-6dab-48a5-97de-1d06b30c4eed tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.226085 Jul 31 01:12:02.166901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 193/394] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:01 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:02.892616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:02.893313 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:02.893820 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:02.894360 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:02.902616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:02.903058 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:02.904886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:02.905344 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:02.906180 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:02.906628 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:02.909880 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:02.910329 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:02.954930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:03.129250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:03.140164 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-bcf4fd09-7e92-47ec-90ca-cf99fd69d7c3 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.251282 Jul 31 01:12:03.140767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 194/395] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:02 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:03.193715 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:03.194432 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:03.201271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:03.201729 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:03.202524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:03.202979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:03.370406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:03.387207 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:03.390342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:03.392602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a7238358-c877-4077-90d2-e7bb2f7e2229 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.202729 Jul 31 01:12:03.393601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 195/396] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:03 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:03.449855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:03.458921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9de3967a-9399-4e7b-b287-88217409fcbc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1557 microversion: 2.1 time: 0.570071 Jul 31 01:12:03.459567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 202/397] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:02 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1557 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:03.482113 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-35b80bd6-4431-49bb-b33f-26b9dd1186a2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:03.482614 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-35b80bd6-4431-49bb-b33f-26b9dd1186a2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:03.489243 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-35b80bd6-4431-49bb-b33f-26b9dd1186a2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:03.489980 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-35b80bd6-4431-49bb-b33f-26b9dd1186a2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:03.495873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-35b80bd6-4431-49bb-b33f-26b9dd1186a2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:03.496450 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-35b80bd6-4431-49bb-b33f-26b9dd1186a2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:03.615053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-35b80bd6-4431-49bb-b33f-26b9dd1186a2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.136517 Jul 31 01:12:03.617075 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 196/398] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:12:03 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf/os-interface => generated 284 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:12:04.165620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:04.167029 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:04.198901 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:04.201362 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:04.203409 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:04.204571 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:04.438746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:04.441099 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:04.455021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:04.456456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:04.459502 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:04.476834 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:04.637534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:04.665990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7a409106-aea1-4652-b883-b08bebd54753 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.506985 Jul 31 01:12:04.679443 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 203/399] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:04 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:05.037076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:05.076326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:05.091113 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:05.093623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-87b0cdd8-927d-4075-a8a0-fdecdf2814ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1744 microversion: 2.20 time: 0.664246 Jul 31 01:12:05.094530 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 197/400] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:04 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1744 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:05.692914 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:05.695645 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:05.717845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:05.735986 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:05.738819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:05.739796 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:05.740429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:05.901024 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-71a75733-f894-48a6-979c-45b452ab258d req-08a41603-3fe3-432f-8c92-2b4072c0ce28 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2b53247d-da0a-46f9-aa9e-f4d09342e5bf", "tag": "b48b5519-311b-4ddf-9138-fa82f7ac77af"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:05.917943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-71a75733-f894-48a6-979c-45b452ab258d req-08a41603-3fe3-432f-8c92-2b4072c0ce28 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:05.918565 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-71a75733-f894-48a6-979c-45b452ab258d req-08a41603-3fe3-432f-8c92-2b4072c0ce28 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:05.935638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-71a75733-f894-48a6-979c-45b452ab258d req-08a41603-3fe3-432f-8c92-2b4072c0ce28 service nova] Creating event network-changed:b48b5519-311b-4ddf-9138-fa82f7ac77af for instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf on hv-cinder-maste Jul 31 01:12:05.945886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-71a75733-f894-48a6-979c-45b452ab258d req-08a41603-3fe3-432f-8c92-2b4072c0ce28 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048663 Jul 31 01:12:05.948852 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 198/401] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:12:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:12:06.136436 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:06.138532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:06.152947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:06.154083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:06.155998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:06.157253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:06.322748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:06.367008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9cf85971-6d41-48b6-af72-173d4a9a034b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.680682 Jul 31 01:12:06.369795 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 204/402] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:05 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 684 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:06.988797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:06.998456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:07.000333 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:07.002004 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4a5671b3-1dd6-457d-9e80-78ad6b6adcca tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1744 microversion: 2.20 time: 0.882532 Jul 31 01:12:07.003283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 199/403] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:06 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1744 bytes in 885 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:07.407802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:07.408697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:07.426622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:07.427494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:07.428561 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:07.429178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:07.690841 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:07.708340 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-92001503-5a95-44a5-8c73-a7dc9f264b7e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.306242 Jul 31 01:12:07.709236 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 205/404] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:07 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:08.044850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:08.046195 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:08.069385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:08.069866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:08.070688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:08.071705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:08.385613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:08.398790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:08.401552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:08.403585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6c3c43fe-c721-42c8-addc-55513f9067b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1744 microversion: 2.20 time: 0.365346 Jul 31 01:12:08.420842 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 200/405] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:08 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1744 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:08.738597 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:08.739992 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:08.760017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:08.761079 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:08.762677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:08.763328 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:09.021215 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:09.042902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9836dd84-384b-4ba8-9602-6a70ce0b6c92 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.314922 Jul 31 01:12:09.044624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 206/406] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:08 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:09.433866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:09.435123 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:09.458147 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:09.460321 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:09.465415 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:09.466574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:09.885037 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:09.907965 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:09.911936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:09.914936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-80e304fe-6a77-4035-8531-79c16ce4c6fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1744 microversion: 2.20 time: 0.486485 Jul 31 01:12:09.916002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 201/407] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:09 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1744 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:10.099381 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:10.101404 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:10.138871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:10.141278 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:10.143776 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:10.154071 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:10.230170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-b1d13604-988b-4bb6-8bf1-2d7187366d99 req-53c42322-5555-467e-b8a0-de5bc6c89a3d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ef19c2df-372c-4c77-ad6c-ada1cd419a14", "name": "network-vif-plugged", "status": "completed", "tag": "f5779c70-0ae8-4a58-a8ba-25f9117da52e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:10.242110 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:10.264308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b1d13604-988b-4bb6-8bf1-2d7187366d99 req-53c42322-5555-467e-b8a0-de5bc6c89a3d service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:10.265487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b1d13604-988b-4bb6-8bf1-2d7187366d99 req-53c42322-5555-467e-b8a0-de5bc6c89a3d service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:10.294852 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-b1d13604-988b-4bb6-8bf1-2d7187366d99 req-53c42322-5555-467e-b8a0-de5bc6c89a3d service nova] Creating event network-vif-plugged:f5779c70-0ae8-4a58-a8ba-25f9117da52e for instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 on hv-cinder-maste Jul 31 01:12:10.308623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-b1d13604-988b-4bb6-8bf1-2d7187366d99 req-53c42322-5555-467e-b8a0-de5bc6c89a3d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091326 Jul 31 01:12:10.310720 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 202/408] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:12:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:12:10.454197 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:10.467510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2a23dc26-9abd-4ceb-934e-9568543ff3e4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.391888 Jul 31 01:12:10.469546 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 207/409] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:10 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:10.956443 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:10.957906 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:10.969862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:10.970785 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:10.972875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:10.973400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:11.206626 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:11.220568 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:11.222950 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:11.224735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-93efe5a1-3040-43e9-99dd-792b4427c478 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1744 microversion: 2.20 time: 0.274472 Jul 31 01:12:11.232556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 203/410] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:10 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1744 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:11.495508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:11.496705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:11.506996 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:11.507963 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:11.510164 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:11.511040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:11.544706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:11.768990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:11.777807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-18642366-711f-4a15-8630-35e73545d5e6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.290313 Jul 31 01:12:11.778584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 208/411] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:11 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:12.256367 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:12.260132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:12.276080 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:12.278297 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:12.280157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:12.281273 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:12.563753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:12.581384 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:12.584080 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:12.586539 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-532447a1-df34-4c96-9f3f-6c037226468f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1744 microversion: 2.20 time: 0.339010 Jul 31 01:12:12.588149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 204/412] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:12 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1744 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:12.807901 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:12.809111 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:12.819768 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:12.820633 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:12.822976 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:12.824126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:13.608481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:13.627797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:13.629424 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:13.635557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-64757103-5d2a-48aa-971c-c774c6e0aaa2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.838483 Jul 31 01:12:13.637082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 209/413] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:12 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:13.652404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:13.652404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:13.652404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:13.652404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:13.867211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:13.879933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:13.882413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:13.884008 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e305ef51-2dfb-4040-baec-2d16b097018f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.260881 Jul 31 01:12:13.884946 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 205/414] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:13 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:14.674664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:14.675294 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:14.684982 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:14.685445 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:14.689824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:14.694731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:14.936878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:14.942757 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:14.956117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:14.959684 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:14.960840 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:14.961460 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:15.205786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:15.220524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:15.222898 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:15.224897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-33922356-0bf3-4897-b93f-67237d7156f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.297601 Jul 31 01:12:15.226100 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 206/415] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:14 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:15.246934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:15.268817 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d4ec777d-d4cd-441f-b2da-51ead04a9a8f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1503 microversion: 2.1 time: 0.598083 Jul 31 01:12:15.283722 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 210/416] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:14 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1503 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:15.315755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d51d6d77-62a5-47a9-89e9-11071b8f300c"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:15.328762 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:15.350237 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:15.351337 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:15.353706 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:15.354893 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:15.437981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] HTTP exception thrown: Cannot 'attach_volume' instance d2206492-6f7a-4a00-ac23-63200f66c940 while it is in vm_state rescued Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Request method failure captured: Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: POST /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940/os-volume_attachments HTTP/1.1 Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 74 Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dZ2ZANicX0K5Ht7oF4q37bz3ry4asTie3_vAAkdfXMm75OrKX_x8Trt62dwbXNsF8s_DZQ_uJMxtIx3O8383U61kM2V-WQSwIut0aag7RRA-YQK94v4Eloyyy8GU4XvoIDt7mCs5Ik6R1dpfM2P_EvHaOxL4bU--FT8De4fXyZLnrw Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:12:15.441787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_3d6a76e846cb48bb91eab4da1c842f2b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 52e5a4deed99442e8eed62f334aa670d Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {"volumeAttachment": {"volumeId": "d51d6d77-62a5-47a9-89e9-11071b8f300c"}} Jul 31 01:12:15.444247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Cannot 'attach_volume' instance d2206492-6f7a-4a00-ac23-63200f66c940 while it is in vm_state rescued Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'server_id': 'd2206492-6f7a-4a00-ac23-63200f66c940', 'body': {'volumeAttachment': {'volumeId': 'd51d6d77-62a5-47a9-89e9-11071b8f300c'}}} Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance d2206492-6f7a-4a00-ac23-63200f66c940 in vm_state rescued. Cannot attach_volume while the instance is in this state. Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:12:15.445429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance d2206492-6f7a-4a00-ac23-63200f66c940 while it is in vm_state rescued Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:12:15.447703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:12:15.449908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:12:15.449908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:12:15.449908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance d2206492-6f7a-4a00-ac23-63200f66c940 while it is in vm_state rescued Jul 31 01:12:15.449908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:12:15.453722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Returning 409 to user: Cannot 'attach_volume' instance d2206492-6f7a-4a00-ac23-63200f66c940 while it is in vm_state rescued {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:12:15.458518 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-819de7f5-437f-4538-a490-4f063692b3b4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.157919 Jul 31 01:12:15.473466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0ad62d6c-1991-48a0-9338-cadf94660690 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:15.474664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0ad62d6c-1991-48a0-9338-cadf94660690 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:15.475935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 207/417] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:12:15 2022] POST /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940/os-volume_attachments => generated 152 bytes in 178 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 31 01:12:15.488358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ad62d6c-1991-48a0-9338-cadf94660690 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:15.489514 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ad62d6c-1991-48a0-9338-cadf94660690 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:15.491008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ad62d6c-1991-48a0-9338-cadf94660690 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:15.491921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ad62d6c-1991-48a0-9338-cadf94660690 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:15.696727 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0ad62d6c-1991-48a0-9338-cadf94660690 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940/action" status: 202 len: 0 microversion: 2.1 time: 0.229055 Jul 31 01:12:15.719204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 211/418] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:12:15 2022] POST /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940/action => generated 0 bytes in 253 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:12:15.722577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:15.725030 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:15.752600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:15.753458 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:15.754489 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:15.755140 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:16.257834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:16.258626 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:16.277034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:16.278058 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:16.278982 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:16.279461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:16.403964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:16.424402 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6043092e-7596-43f0-af72-9f15ee5a63c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1511 microversion: 2.1 time: 0.717139 Jul 31 01:12:16.425013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 208/419] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:15 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1511 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:16.694016 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:16.712786 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:16.717294 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:16.719941 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d298ecb0-1c13-4cf5-bdb5-e5077f65291e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.471241 Jul 31 01:12:16.721313 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 212/420] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:16 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:17.455410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:17.456076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:17.468222 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:17.479940 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:17.481869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:17.482961 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:17.749821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:17.750673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:17.762507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:17.770631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:17.771819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:17.772427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:17.965283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:17.979201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0f13e0e9-c10f-494a-b2db-93a9247af5ff tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1511 microversion: 2.1 time: 0.527931 Jul 31 01:12:17.980485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 209/421] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:17 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1511 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:18.077702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:18.091141 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:18.092829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:18.094070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2f6ae40f-864d-4b23-bd6f-8d5f37ff13ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.348222 Jul 31 01:12:18.103560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 213/422] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:17 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:19.012655 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:19.014311 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:19.023027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:19.024465 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:19.025539 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:19.026159 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:19.151038 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:19.153135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:19.170642 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:19.172814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:19.175223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:19.179490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:19.520075 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:19.554535 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9328b5a8-7e02-426d-ad3e-be95fa464a89 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1511 microversion: 2.1 time: 0.548024 Jul 31 01:12:19.573141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 210/423] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:19 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1511 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:19.606525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:19.636530 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:19.639587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:19.648054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9a63e4e2-b433-4a01-9bda-3e9b27fbeef5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.504826 Jul 31 01:12:19.652418 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 214/424] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:19 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:20.586886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:20.587738 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:20.601466 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:20.602918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:20.604302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:20.610271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:20.679885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:20.683335 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:20.698594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:20.699908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:20.701731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:20.702873 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:21.035284 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:21.050273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:21.052556 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:21.054524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cfd736a4-d8f3-4e92-98ad-2dfa08c82495 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.383130 Jul 31 01:12:21.055958 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 215/425] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:20 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:21.120603 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:21.131636 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a866f825-184c-4859-835c-db9ab7a61d65 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1511 microversion: 2.1 time: 0.548105 Jul 31 01:12:21.132305 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 211/426] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:20 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1511 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:22.079236 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:22.080566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:22.089078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:22.094433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:22.096624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:22.100166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:22.160624 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:22.164019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:22.209408 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:22.214269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:22.224474 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:22.225758 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:22.395047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:22.406673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:22.408997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:22.410630 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-37223e52-c416-4855-8464-888f80882d47 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.335585 Jul 31 01:12:22.412644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 216/427] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:22 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:22.712383 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:22.722842 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:22.734025 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6ce37cd5-8db7-447b-9053-a594a88ed47c tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1517 microversion: 2.1 time: 0.576535 Jul 31 01:12:22.738572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 212/428] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:22 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1517 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:23.466816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:23.467975 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:23.498565 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:23.505906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:23.510813 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:23.516951 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:23.964496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:23.978715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:23.983778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:23.985370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-01c8aba6-a5f3-4ef2-853f-b3f2b0786792 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.525235 Jul 31 01:12:24.010427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 217/429] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:23 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:24.687299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-659ee3d4-35fa-4f85-8948-3b848603f218 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'create', calling method: .version_select at 0x7f3e803a6b80>, body: {"keypair": {"name": "tempest-keypair-1282063812", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH1jmUWJX2WqM463Jx6POAWqq+Xrin4ZME6HAIGo11DpzecQXlvvz48YoeroPIdkFRKuOBAsCT6NygnLISHhqkSnOrXaZb2nNUE7rWUMrg2Qdi9Nrn8NQW/BKRt1EnHWLQ=="}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:24.746881 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-659ee3d4-35fa-4f85-8948-3b848603f218 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:12:24.752161 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-659ee3d4-35fa-4f85-8948-3b848603f218 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for user 52e5a4deed99442e8eed62f334aa670d and project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:12:24.823021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-659ee3d4-35fa-4f85-8948-3b848603f218 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.145378 Jul 31 01:12:24.824185 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 213/430] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:12:24 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:12:25.022233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:25.025298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:25.044318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:25.046349 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:25.050296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:25.051070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:25.343287 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:25.354612 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:25.358293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:25.362060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bb0de49b-8559-488c-b9c3-0f91e9b47123 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.346835 Jul 31 01:12:25.366942 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 218/431] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:25 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:26.416146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:26.418576 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:26.443094 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:26.443716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:26.444772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:26.447318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:27.122714 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:27.155265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:27.165837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:27.168459 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-08516e78-b68d-4530-b51e-2a9e2fa15f00 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.759248 Jul 31 01:12:27.170031 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 214/432] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:26 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:28.232207 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:28.233017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:28.240827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:28.241340 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:28.242185 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:28.252778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:28.710639 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:28.731925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:28.735868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:28.738623 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bddc791c-ca8f-4bb6-a230-dc89f393554f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.511392 Jul 31 01:12:28.740300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 219/433] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:28 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:29.176792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1900014889", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "920eee56-6f5f-4984-b783-e122bc773edb"}], "security_groups": [{"name": "tempest-securitygroup--1815481630"}], "key_name": "tempest-keypair-1282063812", "user_data": "}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:29.789475 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:29.790849 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:29.810664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:29.811131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:29.812001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:29.812465 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:30.081654 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:30.094771 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:30.096405 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:30.097691 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-fac198e9-940e-4f31-bfbc-45e0c47fece0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.312772 Jul 31 01:12:30.102465 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 220/434] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:29 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:31.140921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:31.141914 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:31.155470 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:31.161560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:31.162890 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:31.163537 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:31.482265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:31.491832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:31.494065 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:31.495392 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1cbe2216-f24f-4be4-8a7d-ce367c588d70 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.358806 Jul 31 01:12:31.496064 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 221/435] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:31 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:32.091387 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] validate_networks() for [('920eee56-6f5f-4984-b783-e122bc773edb', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:12:32.523487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:32.525436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:32.539633 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:32.540789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:32.542630 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:32.543927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:32.683097 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:12:32.684008 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:12:32.685006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:12:32.686047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:12:32.687057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:12:32.719849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:32.721095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:32.723861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:32.724788 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:32.763147 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:12:32.768776 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:32.775439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Getting quotas for user 52e5a4deed99442e8eed62f334aa670d and project 3d6a76e846cb48bb91eab4da1c842f2b. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:12:32.783481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:32.785272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:32.786517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0b2b7e3b-3797-4e83-84b5-93ef90472426 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1739 microversion: 2.20 time: 0.271363 Jul 31 01:12:32.799900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 222/436] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:32 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1739 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:32.801572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:12:32.804261 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:12:32.881101 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-231a0e00-ecd5-4eca-a640-8675a7d0b393 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 3.709906 Jul 31 01:12:32.882095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 215/437] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:12:29 2022] POST /compute/v2.1/servers => generated 409 bytes in 3712 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:12:32.906403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d6dd6761-6411-498a-aa74-709a384ffea9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:32.908015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d6dd6761-6411-498a-aa74-709a384ffea9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:33.069060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d6dd6761-6411-498a-aa74-709a384ffea9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:33.095016 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d6dd6761-6411-498a-aa74-709a384ffea9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1246 microversion: 2.1 time: 0.195839 Jul 31 01:12:33.096589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 223/438] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:32 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1246 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:33.810806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:33.811758 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:33.827294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:33.827831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:33.828802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:33.829274 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:34.153964 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:34.155436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:34.171955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:34.180399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:34.183061 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:34.185292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:34.185516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:34.186971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:34.188246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6f818ced-b621-4cde-89df-badd8d0cb0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1721 microversion: 2.20 time: 0.381907 Jul 31 01:12:34.188817 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 216/439] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:33 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1721 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:34.191877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:34.342404 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:34.356489 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-dd79e880-36b5-40e7-b8a1-bf8115e26707 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1242 microversion: 2.1 time: 0.219143 Jul 31 01:12:34.358210 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 224/440] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:34 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1242 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:35.271090 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:35.274487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:35.289470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:35.290260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:35.291979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:35.293215 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:35.385685 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:35.387865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:35.419175 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:35.422290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:35.422290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:35.422290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:35.446768 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:35.526053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:35.537126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7e27ee1d-68e3-4d12-a506-fc1dabc46b59 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1316 microversion: 2.1 time: 0.161241 Jul 31 01:12:35.550975 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 225/441] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:35 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1316 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:35.555399 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:35.571605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:35.573899 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:35.575772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-69d70fcb-323b-43fd-8191-1e142679ac4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1721 microversion: 2.20 time: 0.343855 Jul 31 01:12:35.576947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 217/442] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:35 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1721 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:36.156847 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-66efdc26-71b9-49f2-8b10-c49bf0be64fa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Action: 'create', calling method: .version_select at 0x7f3e80068d30>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-2026137047", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO3Fx2aJHohoGSEbxN00eOGRwgfwUJaDEWzj+ZFMhzpzuOYkDQTgq7FTMRwNZoD28f/nm9Mue8TV3GBU9TMnSbKSt/7dm761Top2ZafjiUzbIW1rmGzZf1E3cY+uMifOwg=="}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:36.164696 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=46 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:36.179704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-66efdc26-71b9-49f2-8b10-c49bf0be64fa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Getting quotas for project b018ba41816049feaa75e10489ec5281. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:12:36.185971 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-66efdc26-71b9-49f2-8b10-c49bf0be64fa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Getting quotas for user 1776c123f98c44b69807ec2b0281b718 and project b018ba41816049feaa75e10489ec5281. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:12:36.217106 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-66efdc26-71b9-49f2-8b10-c49bf0be64fa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.064391 Jul 31 01:12:36.218617 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 226/443] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:12:36 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:12:36.287577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-75784612", "imageRef": "197667d6-90cd-49dc-8ccc-b4d2a239c5ed", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-2026137047", "networks": [{"uuid": "d0800c46-2b27-4b24-b275-93ca1c973007"}]}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:36.372665 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] validate_networks() for [('d0800c46-2b27-4b24-b275-93ca1c973007', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:12:36.584028 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:36.588668 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:36.600773 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:36.602241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:36.604146 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:36.604966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:36.734776 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:36.745264 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-efe3b097-f40b-4db5-9ce4-9f3c74fe714d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1304 microversion: 2.1 time: 0.179115 Jul 31 01:12:36.745968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 227/444] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:36 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1304 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:36.756140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:36.757425 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:36.768426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:36.770085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:36.771112 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:36.771796 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:36.966214 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:36.992390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:36.997725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:37.001913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-74836313-0b22-4e23-ad8a-83f10c5a44c7 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1721 microversion: 2.20 time: 0.253133 Jul 31 01:12:37.005908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 228/445] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:36 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1721 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:37.183100 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:12:37.183841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:12:37.184849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:12:37.185505 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:12:37.186190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:12:37.200126 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:37.200847 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:37.201858 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:37.202456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:37.236900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-a62156f8-5b48-48b5-b34f-9d3ab42d1029 req-b738037a-0f70-4aff-9606-b17f6733ac46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ef19c2df-372c-4c77-ad6c-ada1cd419a14", "name": "network-vif-unplugged", "status": "completed", "tag": "f5779c70-0ae8-4a58-a8ba-25f9117da52e"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:37.244613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Getting quotas for project b018ba41816049feaa75e10489ec5281. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:12:37.250894 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Getting quotas for user 1776c123f98c44b69807ec2b0281b718 and project b018ba41816049feaa75e10489ec5281. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:12:37.259976 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a62156f8-5b48-48b5-b34f-9d3ab42d1029 req-b738037a-0f70-4aff-9606-b17f6733ac46 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:37.261131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a62156f8-5b48-48b5-b34f-9d3ab42d1029 req-b738037a-0f70-4aff-9606-b17f6733ac46 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:37.268372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:12:37.272554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='197667d6-90cd-49dc-8ccc-b4d2a239c5ed',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:12:37.297657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-a62156f8-5b48-48b5-b34f-9d3ab42d1029 req-b738037a-0f70-4aff-9606-b17f6733ac46 service nova] Creating event network-vif-unplugged:f5779c70-0ae8-4a58-a8ba-25f9117da52e for instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 on hv-cinder-maste Jul 31 01:12:37.314833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-a62156f8-5b48-48b5-b34f-9d3ab42d1029 req-b738037a-0f70-4aff-9606-b17f6733ac46 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.083118 Jul 31 01:12:37.317547 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 229/446] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:12:37 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:12:37.381473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f2712420-8949-49e8-af8c-381b77b8f708 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.102524 Jul 31 01:12:37.387175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 218/447] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:12:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 1109 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:12:37.405128 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-61b1988d-429a-4d5c-b0e9-75e53dba8ea8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:37.408577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-61b1988d-429a-4d5c-b0e9-75e53dba8ea8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:37.550618 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-61b1988d-429a-4d5c-b0e9-75e53dba8ea8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:37.572463 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-61b1988d-429a-4d5c-b0e9-75e53dba8ea8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1257 microversion: 2.1 time: 0.174439 Jul 31 01:12:37.574168 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 230/448] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:37 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1257 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:37.776021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:37.777098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:37.790133 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:37.790910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:37.791873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:37.792556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.038252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:38.040485 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:38.055481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.056295 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.058324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.060171 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.121865 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:38.136043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6e30e908-1f7a-4eaa-a5da-ebd7a0af33d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.366070 Jul 31 01:12:38.138204 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 219/449] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:37 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:38.264361 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:38.276059 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:38.278705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:38.280910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-32cf61a0-cf7a-45d0-a829-d0ea877e8f98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1664 microversion: 2.20 time: 0.253406 Jul 31 01:12:38.281846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 231/450] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:38 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1664 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:38.315196 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5f586b7f-0508-4a02-9b4b-0e06352abf7f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f784b82c-57ad-4c26-a15f-75cea76d79c3"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:38.320495 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5f586b7f-0508-4a02-9b4b-0e06352abf7f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:38.331183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f586b7f-0508-4a02-9b4b-0e06352abf7f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.332080 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f586b7f-0508-4a02-9b4b-0e06352abf7f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.333583 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f586b7f-0508-4a02-9b4b-0e06352abf7f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.334325 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f586b7f-0508-4a02-9b4b-0e06352abf7f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.598891 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:38.599826 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:38.610616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.611429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.612824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.613610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.697571 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:38.712014 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d8a65b4a-cb83-4d8d-9a9b-d199997d1b30 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1261 microversion: 2.1 time: 0.120773 Jul 31 01:12:38.712957 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 232/451] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:38 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1261 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:38.718145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5f586b7f-0508-4a02-9b4b-0e06352abf7f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.410908 Jul 31 01:12:38.719390 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 220/452] 10.222.0.42 () {62 vars in 1364 bytes} [Sun Jul 31 01:12:38 2022] POST /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments => generated 188 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:12:38.877114 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:38.878120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:38.887677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.888293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.889110 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:38.889725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:38.959622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=117943) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Jul 31 01:12:39.042362 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6834afc8-a93e-4695-b5af-44bc7b84acf5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/action" status: 202 len: 0 microversion: 2.20 time: 0.171971 Jul 31 01:12:39.044325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 233/453] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Jul 31 01:12:38 2022] POST /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/action => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:12:39.053284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:39.054037 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:39.062345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:39.063062 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:39.064018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:39.064754 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:39.171978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:39.173026 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:39.184649 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:39.185938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:39.187343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:39.188123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:39.270871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:39.283190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:39.284870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:39.286398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0940bf0f-dd4f-4705-a49b-88aea9eba238 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1750 microversion: 2.20 time: 0.237316 Jul 31 01:12:39.287228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 221/454] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:39 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1750 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:39.373425 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:39.381443 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-36f998a6-443a-415c-88ad-f1c1cdece7df tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.214493 Jul 31 01:12:39.382826 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 234/455] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:39 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:39.741389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:39.743234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:39.759663 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:39.760646 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:39.762125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:39.762874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:40.148098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:40.162759 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-34cb012a-b748-4a7b-95a6-dd4d2794ebe6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1261 microversion: 2.1 time: 0.425598 Jul 31 01:12:40.165591 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 222/456] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:39 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1261 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:40.320941 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:40.322246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:40.331861 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:40.333012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:40.334673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:40.335779 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:40.409920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:40.410964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:40.425066 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=7 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:40.433713 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:40.434679 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:40.435610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:40.436055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:40.585188 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:40.601523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:40.603718 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:40.605185 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-233d95b0-5118-467b-8154-7b6068509d5e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1750 microversion: 2.20 time: 0.290097 Jul 31 01:12:40.605859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 235/457] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:40 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1750 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:40.700220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:40.713808 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a77f0dd1-3f6f-4a47-bb68-439ddd6b4173 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.310484 Jul 31 01:12:40.715920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 223/458] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:40 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:41.000577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-6ba8d0b0-b316-42c4-821c-a523dff2aade req-1a661689-f599-4360-9cae-f64f694d8abd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4a6e0502-05f5-40cb-98d9-4d0fc3d3c656", "tag": "ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:41.018855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-6ba8d0b0-b316-42c4-821c-a523dff2aade req-1a661689-f599-4360-9cae-f64f694d8abd service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:41.019694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-6ba8d0b0-b316-42c4-821c-a523dff2aade req-1a661689-f599-4360-9cae-f64f694d8abd service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:41.044372 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-6ba8d0b0-b316-42c4-821c-a523dff2aade req-1a661689-f599-4360-9cae-f64f694d8abd service nova] Creating event network-changed:ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc for instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 on hv-cinder-maste Jul 31 01:12:41.050305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-6ba8d0b0-b316-42c4-821c-a523dff2aade req-1a661689-f599-4360-9cae-f64f694d8abd service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056362 Jul 31 01:12:41.051276 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 236/459] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:12:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:12:41.186010 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:41.186618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:41.193219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:41.193712 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:41.194556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:41.195029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:41.288646 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:41.299877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-36e2a582-053a-4d49-95ba-3a05c1859c68 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1261 microversion: 2.1 time: 0.117391 Jul 31 01:12:41.300826 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 224/460] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:41 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1261 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:41.634928 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:41.636473 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:41.650075 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:41.651993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:41.656090 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:41.656090 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:41.746432 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:41.748347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:41.766554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:41.770639 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:41.773656 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:41.775728 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:41.943317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:41.969709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:41.973428 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:41.975922 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-21ae4f94-b6a0-4c41-b880-83d02c93f434 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1750 microversion: 2.20 time: 0.346049 Jul 31 01:12:41.978504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 237/461] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:41 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1750 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:42.084168 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:42.092837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8fd3ecb2-e103-404e-9e5f-5d0cdfb2fec8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.355531 Jul 31 01:12:42.093669 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 225/462] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:41 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:42.328747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:42.330126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:42.337900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:42.338640 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:42.339981 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:42.340760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:42.433128 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:42.449052 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-419a7fa6-1b77-477c-96ac-0a1af6081683 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1309 microversion: 2.1 time: 0.123797 Jul 31 01:12:42.450458 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 238/463] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:42 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1309 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:43.023384 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:43.027609 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:43.049683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:43.050455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:43.051846 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:43.052613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:43.125012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:43.130964 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:43.170803 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:43.173696 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:43.176288 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:43.181092 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:43.628858 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:43.645579 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:43.648704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:43.650635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0228bdf5-5e89-44d0-945d-0e303277755e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1752 microversion: 2.20 time: 0.635216 Jul 31 01:12:43.652347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 226/464] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:43 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1752 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:43.674339 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:43.680040 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:43.715310 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:43.716213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:43.717785 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:43.723590 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:43.888088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:43.903698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e7d588c7-02ea-48fd-acbd-7085efd11b73 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1315 microversion: 2.1 time: 0.249759 Jul 31 01:12:43.904859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 227/465] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:43 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1315 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:44.100142 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:44.112209 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8ea46a46-0dc4-432e-bc19-373a848d70a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.999547 Jul 31 01:12:44.113071 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 239/466] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:43 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 1001 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:44.715125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:44.715825 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:44.723216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:44.723757 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:44.724601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:44.725087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:44.882664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:44.895552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:44.898370 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:44.900399 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6f85f516-26a4-4c95-bc90-9037d5f78f8e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.189228 Jul 31 01:12:44.901244 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 228/467] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:44 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:44.938943 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:44.939807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:44.948217 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:44.948891 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:44.949856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:44.950448 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:45.117743 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:45.125913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-061b68b9-1704-4ef6-b691-a1063c1ec9ed tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.192365 Jul 31 01:12:45.126524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 240/468] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:44 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:45.138418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:45.139528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:45.150798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:45.151624 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:45.153051 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:45.153833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:45.162162 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:45.341006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:45.349940 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cd718f76-e323-45fa-b242-7ffc9b9d3474 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.218043 Jul 31 01:12:45.350821 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 229/469] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:45 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:45.932332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:45.933142 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:45.940051 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:45.941126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:45.941814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:45.942430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:46.103143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:46.117896 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:46.121638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:46.123868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6d74fd64-f9a1-4a0b-904b-8a41d0a0dfa5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.196018 Jul 31 01:12:46.125112 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 241/470] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:45 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:46.154600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:46.155685 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:46.170569 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:46.171864 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:46.173396 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:46.174306 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:46.377404 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:46.378951 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:46.397472 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:46.398959 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:46.401221 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:46.402680 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:46.480426 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:46.497938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-59e90a36-9d42-4cf2-a44c-d78885085ac0 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.351008 Jul 31 01:12:46.511966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 230/471] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:46 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:46.650161 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:46.662826 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f48e6bb1-3c6f-4e60-b103-349f70608cb2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.292607 Jul 31 01:12:46.664644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 242/472] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:46 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:47.165270 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:47.166463 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:47.174913 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:47.175554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:47.176689 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:47.177347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:47.224805 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=4 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:47.260226 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=16 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:47.492248 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:47.510386 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:47.513086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:47.520085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d108c98d-e55f-4ad6-a4ba-b07f772c77c0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.360182 Jul 31 01:12:47.520601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 231/473] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:47 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:47.535536 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:47.537487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:47.580827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=8 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:47.583709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:47.584516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:47.585940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:47.594273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:47.686252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:47.686878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:47.703169 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:47.703660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:47.704454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:47.704896 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:47.942973 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:47.952290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c507dad2-d20f-4c84-81ed-5cbc17bfa0ca tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.423227 Jul 31 01:12:47.953161 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 243/474] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:47 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:48.012328 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:48.040209 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-93f88549-82e7-4abe-91b6-77d360d792a9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.357333 Jul 31 01:12:48.041730 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 232/475] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:47 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:48.570358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:48.573369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:48.585446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:48.586195 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:48.587557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:48.588605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:48.594342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:12:48.881047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:48.899786 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:48.904667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:48.907374 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-907e0ce5-9b3d-4cd5-b97b-07b3e9a15d13 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.343577 Jul 31 01:12:48.908877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 244/476] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:48 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:48.981771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:48.986511 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:48.998737 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:49.000135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:49.002359 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:49.003783 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:49.078220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:49.079673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:49.099422 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:49.101176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:49.103174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:49.104115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:49.343291 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:49.351850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-30a5690f-37e6-4ffc-8848-eb9441d89c85 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.379018 Jul 31 01:12:49.353694 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 233/477] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:48 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:49.515517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:49.524654 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-dcf30cf1-c00b-4805-9575-14f80dce0ec0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.456902 Jul 31 01:12:49.525263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 245/478] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:49 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:49.888791 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-87a9e55c-f3c5-479f-ab12-04870873eeaf req-ea2aae53-db6d-4f35-95e6-601dc8c40338 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4a6e0502-05f5-40cb-98d9-4d0fc3d3c656", "name": "network-vif-plugged", "status": "completed", "tag": "ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:12:49.941679 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:49.942829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:49.950258 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-87a9e55c-f3c5-479f-ab12-04870873eeaf req-ea2aae53-db6d-4f35-95e6-601dc8c40338 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:49.951092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-87a9e55c-f3c5-479f-ab12-04870873eeaf req-ea2aae53-db6d-4f35-95e6-601dc8c40338 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:49.967809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:49.974557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:49.989446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:49.990312 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:50.009092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-87a9e55c-f3c5-479f-ab12-04870873eeaf req-ea2aae53-db6d-4f35-95e6-601dc8c40338 service nova] Creating event network-vif-plugged:ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc for instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 on hv-cinder-maste Jul 31 01:12:50.031556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-87a9e55c-f3c5-479f-ab12-04870873eeaf req-ea2aae53-db6d-4f35-95e6-601dc8c40338 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.150423 Jul 31 01:12:50.035537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 234/479] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:12:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:12:50.356176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:50.380674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:50.393540 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:50.394697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:50.397137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f5901c03-b488-47e3-9d6e-99457e923686 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.463479 Jul 31 01:12:50.398460 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:50.405801 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 246/480] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:49 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:50.410874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:50.411734 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:50.413326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:50.414055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:50.550483 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:50.552471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:50.566743 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:50.567691 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:50.569077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:50.571284 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:50.875050 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:50.890850 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f0309df3-3071-43cd-b45e-f7e72b095ddd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.503589 Jul 31 01:12:50.908405 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 235/481] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:50 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:50.945358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:50.957357 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-af03680a-7575-425c-8c8e-6bd90ee922e7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.413749 Jul 31 01:12:50.957964 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 247/482] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:50 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:51.437608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:51.438746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:51.456564 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:51.458580 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:51.461099 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:51.461651 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:51.744743 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:51.769973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:51.773097 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:51.775494 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-549d5ece-a44a-44af-9b40-c2bd400e7638 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.343996 Jul 31 01:12:51.781812 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 236/483] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:51 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:12:51.936223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:51.937583 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:51.970676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:51.971855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:51.973167 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:51.974308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:51.993869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:51.994912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:52.023692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:52.023692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:52.023692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:52.023692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:52.353994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:52.369385 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-62bf6dd8-fbb4-47e4-be6e-097e8470c671 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.446722 Jul 31 01:12:52.389885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 248/484] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:51 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:52.441076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:52.456414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-87844d3b-436a-49c4-901d-109aaeb0991f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.469590 Jul 31 01:12:52.478585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 237/485] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:51 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:52.815017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:52.815699 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:52.839075 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:52.839610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:52.840445 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:52.840901 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:53.133411 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:53.153345 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:53.157305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:53.160162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-00bbed9e-89f8-4838-b039-c650cb13c95c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.349600 Jul 31 01:12:53.161693 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 249/486] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:52 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:53.405754 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:53.407952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:53.435804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:53.437444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:53.442951 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:53.448734 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:53.495790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:53.502007 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:53.522913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:53.528589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:53.531205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:53.536774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:53.935021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:53.965478 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e8143719-22ec-40a9-8bab-90e4eac28b42 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.481072 Jul 31 01:12:53.966390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 250/487] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:53 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:54.188851 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:54.190399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:54.196033 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:54.201406 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:54.203889 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:54.206969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:54.212139 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:54.223562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-27d219d3-2f32-4f3b-87e4-311bb89447b4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.822255 Jul 31 01:12:54.229158 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 238/488] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:53 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:54.487736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:54.512522 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:54.516088 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:54.518738 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f9644fef-32fc-483f-b57b-a8b08b911ed9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.335365 Jul 31 01:12:54.520317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 251/489] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:54 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:55.008958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:55.011622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:55.028092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:55.029467 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:55.031554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:55.032689 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:55.268070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:55.272394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:55.288925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:55.290243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:55.292099 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:55.293356 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:55.539749 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:55.557416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3cd55dd9-8a41-4257-8bca-332011fc972e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.312054 Jul 31 01:12:55.558524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 252/490] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:55 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:55.567591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:55.568521 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:55.578213 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:55.578946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:55.580118 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:55.580734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:55.682455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:55.697401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8a772a56-a49c-40da-8e65-da0c98a0d0b7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.699130 Jul 31 01:12:55.698348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 239/491] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:54 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:55.912350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:55.922866 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:55.924469 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:55.925687 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d625e3b5-1097-4a72-8691-fec8a079a96e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.362859 Jul 31 01:12:55.926264 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 253/492] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:55 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:56.604959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:56.606295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:56.616891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:56.617718 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:56.619095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:56.619849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:56.740148 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:56.743446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:56.758532 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:56.760478 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:56.763500 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:56.764681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:56.860461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:56.876026 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-bfbdf67b-daa6-449d-b050-67614bb42b40 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.275303 Jul 31 01:12:56.891393 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 240/493] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:56 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:56.954821 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:56.955121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:56.966302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:56.967102 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:56.968611 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:56.969701 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:57.098909 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:57.110704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-09cb322e-4a0e-4c27-b05d-5426837f8859 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.378201 Jul 31 01:12:57.111994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 254/494] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:56 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:57.719629 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:57.730483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:57.732947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:57.734954 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-59d284b3-08cc-45b7-96cb-179f339f9d40 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.788411 Jul 31 01:12:57.735909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 241/495] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:56 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:57.911775 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:57.914040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:57.926745 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:57.927119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:57.928235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:57.930151 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:58.152007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:58.153555 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:58.165330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:58.166465 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:58.168526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:58.169823 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:58.328512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:58.351695 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1403906a-9529-4698-bbcc-49b66030a4f8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.446838 Jul 31 01:12:58.363407 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 255/496] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:57 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:12:58.420323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:58.434422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-73c7ef8d-046c-4e76-bd3e-dfed200a790d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.288891 Jul 31 01:12:58.435073 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 242/497] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:58 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:12:58.771737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:58.777461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:58.814528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:58.817791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:58.823224 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:58.848331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:59.413023 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:59.453190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:59.465530 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:59.466700 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:59.471678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:59.488431 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:59.685351 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:59.704473 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:59.706119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:12:59.707412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-846488a8-db5e-4419-8337-d0190840fc82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.951212 Jul 31 01:12:59.708016 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 256/498] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:12:58 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:12:59.727998 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:12:59.728632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:12:59.735543 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:59.735992 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:12:59.737398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:12:59.738043 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:00.096910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:00.118711 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4cb93052-2118-4d09-aa3f-8189c8d4daa2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.727657 Jul 31 01:13:00.119948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 243/499] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:59 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:00.501261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:00.537945 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-09192467-9cff-4376-8dc1-298ea3d8f194 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.828428 Jul 31 01:13:00.540953 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 257/500] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:12:59 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 832 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:00.758523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:00.760245 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-d3e7d4bb-53ad-47ef-8341-705238ca7ad1 req-af735c13-d121-4919-981a-a69876b6e65c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7a3b72f2-b29c-4115-b742-1fa4a86e06b6", "tag": "f704de2f-7bdb-4ef2-80b7-29cb6b9472eb"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:00.760935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:00.777193 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-d3e7d4bb-53ad-47ef-8341-705238ca7ad1 req-af735c13-d121-4919-981a-a69876b6e65c service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:00.777938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-d3e7d4bb-53ad-47ef-8341-705238ca7ad1 req-af735c13-d121-4919-981a-a69876b6e65c service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:00.778817 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:00.780753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:00.783956 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:00.785055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:00.801349 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-d3e7d4bb-53ad-47ef-8341-705238ca7ad1 req-af735c13-d121-4919-981a-a69876b6e65c service nova] Creating event network-changed:f704de2f-7bdb-4ef2-80b7-29cb6b9472eb for instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 on hv-cinder-maste Jul 31 01:13:00.811087 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-d3e7d4bb-53ad-47ef-8341-705238ca7ad1 req-af735c13-d121-4919-981a-a69876b6e65c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057544 Jul 31 01:13:00.812575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 258/501] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:00.971814 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:00.982338 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:00.983949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:00.985207 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7cd7e808-488a-4e55-8615-32e2737fdd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.239504 Jul 31 01:13:00.985797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 244/502] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:00 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:01.157349 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:01.162020 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:01.179639 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:01.180043 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:01.181634 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:01.182359 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:01.525575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:01.536317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-38fbe701-75bc-4e41-a634-320c0bc1c44d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.388564 Jul 31 01:13:01.537313 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 259/503] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:01 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:01.569230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:01.571786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:01.583771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:01.585205 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:01.586738 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:01.587666 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:01.822587 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:01.839143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-aa3f4bec-2c7c-43bd-929e-675580c43aa0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1372 microversion: 2.1 time: 0.276420 Jul 31 01:13:01.842989 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 245/504] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:01 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1372 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:02.010564 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:02.012069 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:02.025178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:02.026056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:02.027537 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:02.031709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:02.392512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:02.411097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:02.415876 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:02.418228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-06c4bc0b-5a96-4403-9ada-1ca67e0b8fb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.411605 Jul 31 01:13:02.419621 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 260/505] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:02 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:02.584012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:02.590326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:02.622252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:02.622483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:02.623839 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:02.634260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:02.702454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:02.875187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:02.876558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:02.898869 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:02.899636 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:02.907924 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:02.908811 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:02.985296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:03.174131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:03.209532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-49adb0eb-326f-44a5-9777-87d051ed4c0f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.638440 Jul 31 01:13:03.212905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 246/506] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:02 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:03.442444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:03.443004 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:03.459457 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:03.459922 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:03.460733 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:03.461180 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:03.824201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:03.841753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:03.845261 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:03.848110 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7272f51e-9618-4f2b-be60-10d8140411ba tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.409356 Jul 31 01:13:03.849493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 247/507] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:03 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:03.999395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:04.017837 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ac8568d5-29d0-4ff1-b962-7a57c89e6272 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1566 microversion: 2.1 time: 1.155898 Jul 31 01:13:04.020024 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 261/508] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:02 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1566 bytes in 1159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:04.050016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c8e912e6-c2f7-4541-8051-2544fa6545c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:04.057481 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c8e912e6-c2f7-4541-8051-2544fa6545c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:04.069984 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c8e912e6-c2f7-4541-8051-2544fa6545c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:04.071660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c8e912e6-c2f7-4541-8051-2544fa6545c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:04.073012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c8e912e6-c2f7-4541-8051-2544fa6545c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:04.073693 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c8e912e6-c2f7-4541-8051-2544fa6545c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:04.252711 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:04.256656 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:04.278879 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:04.286652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:04.288430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:04.289491 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:04.404215 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c8e912e6-c2f7-4541-8051-2544fa6545c9 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.362272 Jul 31 01:13:04.409454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 248/509] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:13:04 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-interface => generated 284 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:13:04.877284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:04.878161 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:04.899483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:04.902924 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:04.904445 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:04.905231 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:05.458915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:05.484371 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ffb0545b-c947-4083-b9e1-4fb520ef3b2a tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 1.251915 Jul 31 01:13:05.498641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 262/510] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:04 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 1268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:05.826035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:05.855767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:05.866481 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:05.869011 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f01e26bc-452b-47cf-8057-43c3a1bd3a15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.998912 Jul 31 01:13:05.870101 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 249/511] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:04 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 1001 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:06.506981 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:06.507824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:06.510400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:06.525692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:06.527802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:06.531573 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:06.532459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:06.908805 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:06.915536 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:06.916178 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:06.929691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:06.930167 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:06.931266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:06.931761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:06.933141 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ffce8621-fd6e-419b-ade6-d5bc44215b5b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.430933 Jul 31 01:13:06.934915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 263/512] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:06 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:06.944428 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-ad553d53-10a1-4306-b513-ad2c68f91e6f req-c9909b96-68a6-48af-bdf6-725da64cbdc2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4a6e0502-05f5-40cb-98d9-4d0fc3d3c656", "tag": "ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:06.977072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-ad553d53-10a1-4306-b513-ad2c68f91e6f req-c9909b96-68a6-48af-bdf6-725da64cbdc2 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:06.978273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-ad553d53-10a1-4306-b513-ad2c68f91e6f req-c9909b96-68a6-48af-bdf6-725da64cbdc2 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:07.013045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-ad553d53-10a1-4306-b513-ad2c68f91e6f req-c9909b96-68a6-48af-bdf6-725da64cbdc2 service nova] Creating event network-changed:ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc for instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 on hv-cinder-maste Jul 31 01:13:07.023665 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-ad553d53-10a1-4306-b513-ad2c68f91e6f req-c9909b96-68a6-48af-bdf6-725da64cbdc2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086318 Jul 31 01:13:07.024760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 264/513] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:07.222594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:07.246223 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:07.249880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:07.252319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-da6ced73-5b07-4c07-9df0-306f12262a6f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.341035 Jul 31 01:13:07.253709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 250/514] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:06 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:07.970974 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:07.971960 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:08.000301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:08.001484 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:08.003948 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:08.005627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:08.278502 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:08.280122 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:08.288114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:08.289086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:08.290178 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:08.290880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:08.364442 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:08.393128 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ed76b737-90c3-48b6-b366-1aaaf592e3df tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.427230 Jul 31 01:13:08.403801 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 265/515] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:07 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:08.741553 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:08.756881 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:08.761549 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:08.767437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-62bc19c1-4049-4cdb-b213-95208ab6387f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.492828 Jul 31 01:13:08.769295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 251/516] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:08 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:09.430799 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:09.432429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:09.475596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:09.476782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:09.480437 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:09.481490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:09.794395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:09.795014 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:09.812988 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:09.813450 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:09.814281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:09.821766 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:09.877193 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:09.911343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-533e537b-cb68-47e0-b0e2-5992c23d3eeb tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.478541 Jul 31 01:13:09.913233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 266/517] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:09 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:10.419888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:10.437345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:10.439181 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:10.440531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3ca3c6a6-459e-4be3-be73-8f0b1e3c1910 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.650871 Jul 31 01:13:10.442847 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 252/518] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:09 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:10.938534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:10.944363 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:10.976790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:10.978004 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:10.979784 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:10.980854 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:11.465651 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:11.467789 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:11.480313 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=60 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:11.486804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:11.486804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:11.495308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:11.499048 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:11.611060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:11.631489 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ed844ddf-11a7-43bd-a951-d84b57972eee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.701323 Jul 31 01:13:11.632739 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 267/519] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:10 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 704 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:12.144320 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:12.206378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:12.209561 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:12.211838 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c2f06701-da9a-446a-9900-80014ac34c4c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.750136 Jul 31 01:13:12.225485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 253/520] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:11 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 765 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:12.653242 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:12.654195 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:12.670220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:12.674481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:12.686814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:12.688633 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:12.875047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-3b4c6ec5-d8e0-4e6b-95a3-7e0de2593508 req-3ced578a-2fba-4326-9c24-7de255d5b782 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7a3b72f2-b29c-4115-b742-1fa4a86e06b6", "name": "network-vif-plugged", "status": "completed", "tag": "f704de2f-7bdb-4ef2-80b7-29cb6b9472eb"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:12.897078 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-3b4c6ec5-d8e0-4e6b-95a3-7e0de2593508 req-3ced578a-2fba-4326-9c24-7de255d5b782 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:12.897650 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-3b4c6ec5-d8e0-4e6b-95a3-7e0de2593508 req-3ced578a-2fba-4326-9c24-7de255d5b782 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:12.932547 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-3b4c6ec5-d8e0-4e6b-95a3-7e0de2593508 req-3ced578a-2fba-4326-9c24-7de255d5b782 service nova] Creating event network-vif-plugged:f704de2f-7bdb-4ef2-80b7-29cb6b9472eb for instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 on hv-cinder-maste Jul 31 01:13:12.953224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-3b4c6ec5-d8e0-4e6b-95a3-7e0de2593508 req-3ced578a-2fba-4326-9c24-7de255d5b782 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083661 Jul 31 01:13:12.954564 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 254/521] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:13:13.087390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:13.099978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4dc4b695-c2c7-4c55-adfa-74d96a201ac6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.450445 Jul 31 01:13:13.116107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 268/522] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:12 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:13.189981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-052a6ba8-4960-43f9-8350-af4035e20010 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Action: 'create', calling method: .version_select at 0x7f3e81941790>, body: {"keypair": {"name": "tempest-TestShelveInstance-1706218805", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN3f8ddzO/A8wlr5rmk3Kav5bM4M99SQd9t1e/aYsTVbEL4iAz58YNGwYGEFrPlhYARcsabKA2mAWtNtmReCoJmvY+2DRLvVjlrJDobMx1xtOydzX2+yHRV67pGPNbmjkg=="}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:13.217234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-052a6ba8-4960-43f9-8350-af4035e20010 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Getting quotas for project 5ca9597665cd4068b0a111136d6e0489. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:13:13.229021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-052a6ba8-4960-43f9-8350-af4035e20010 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Getting quotas for user cb4bfd7ea6664747a6e0693a2fd11e3b and project 5ca9597665cd4068b0a111136d6e0489. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:13:13.244034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:13.244838 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:13.267161 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:13.268237 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:13.269357 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:13.270275 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:13.280494 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-052a6ba8-4960-43f9-8350-af4035e20010 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.321906 Jul 31 01:13:13.288535 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 255/523] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:13:12 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:13.504252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:13.521372 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:13.530368 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:13.533490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-81e88c04-ea74-440d-9d5d-5ce5fdcaa0b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.292321 Jul 31 01:13:13.551648 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 269/524] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:13 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:14.128074 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:14.129903 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:14.153365 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:14.162598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:14.164664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:14.169480 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:14.558815 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:14.559756 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:14.563265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:14.569811 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:14.570698 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:14.571792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:14.572455 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:14.573006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8b5a5f55-3df4-460a-a34d-fe71533e8ba9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.450917 Jul 31 01:13:14.596688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 256/525] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:14 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:14.829557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:14.856049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:14.859626 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:14.861944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9bfde156-9494-4d54-bfc6-bf5437575afd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.307233 Jul 31 01:13:14.864000 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 270/526] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:14 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:15.603072 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:15.606375 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:15.636528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:15.636528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:15.638258 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:15.639021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:15.887980 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:15.898736 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1e29d031-3125-4973-a88b-c82ffc819075 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.305368 Jul 31 01:13:15.899387 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 257/527] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:15 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:15.914246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:15.918060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:15.943271 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:15.948616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:15.956902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:15.958452 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:16.293390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:16.314125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:16.317229 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:16.320398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f7b9e063-4ac3-4015-a597-0464a623a6f5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.415867 Jul 31 01:13:16.321395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 271/528] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:15 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:16.922617 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:16.923908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:16.933692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:16.934058 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:16.935072 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:16.935728 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:17.166490 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:17.178435 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-bb83c8a0-579d-41b2-bf55-80d3bf9cbafd tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.259392 Jul 31 01:13:17.179575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 258/529] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:16 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:17.365249 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:17.368496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:17.400342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:17.401019 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:17.407331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:17.407908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:17.701439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:17.713216 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:17.716108 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:17.717683 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3fc9c447-009a-424f-ab24-0a4bead34b25 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.360003 Jul 31 01:13:17.718468 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 272/530] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:17 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:18.204542 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:18.205183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:18.214806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:18.215382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:18.216552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:18.217172 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:18.544201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:18.559417 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-abf8f9d7-e1f5-4d98-ade7-5f5f28814f61 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.362132 Jul 31 01:13:18.561002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 259/531] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:18 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:18.763683 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:18.764435 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:18.772261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:18.773062 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:18.774203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:18.774909 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:19.002216 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:19.015840 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:19.017870 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:19.019560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-05f9ffa4-2eb6-42b2-9d8c-3c896d531a18 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.260532 Jul 31 01:13:19.020467 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 273/532] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:18 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:19.592431 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:19.593731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:19.606734 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:19.607705 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:19.609271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:19.610086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:19.782424 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:19.791844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7966032f-d9bf-4e58-97ee-430cb725d2d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.212302 Jul 31 01:13:19.792454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 260/533] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:19 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:20.060779 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:20.061395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:20.069129 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:20.069582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:20.070367 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:20.070806 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:20.274506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:20.284213 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:20.285822 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:20.287475 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-53db69a2-d3d7-4ab9-b751-1ba824fb7ec0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.230750 Jul 31 01:13:20.288076 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 274/534] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:20 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:20.813512 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:20.814456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:20.821238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:20.821697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:20.822492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:20.822937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:21.020155 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:21.028726 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-37ef1707-8537-4f6e-921c-d721e9525a6f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.219315 Jul 31 01:13:21.029324 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 261/535] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:20 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:21.311672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:21.312658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:21.320658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:21.320980 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:21.321780 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:21.322223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:21.485898 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:21.501571 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:21.504586 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:21.507312 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a2a0b6fd-c951-4ac4-9a8b-4ac865038dc3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.199421 Jul 31 01:13:21.508334 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 275/536] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:21 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:22.050981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:22.051806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:22.059154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:22.059648 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:22.060448 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:22.060889 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:22.232002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:22.246754 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fe9ee1c2-c803-4735-be44-c23c1d54ad10 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.199926 Jul 31 01:13:22.247592 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 262/537] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:22 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:22.696320 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:22.696982 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:22.703725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:22.704421 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:22.705393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:22.705943 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:22.874647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:22.888736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:22.890634 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:22.891903 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c4b6ec2b-c1e4-46ad-8326-5e616e3f2d42 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.351946 Jul 31 01:13:22.892468 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 276/538] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:22 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:23.273454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:23.274652 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:23.278400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=4 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:23.288627 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:23.289411 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:23.290786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:23.291684 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:23.444216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:23.453462 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d054e434-4ecd-4227-81ed-56de199d8ddf tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.187575 Jul 31 01:13:23.453462 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 263/539] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:23 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:23.931577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:23.932598 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:23.940505 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:23.941110 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:23.942003 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:23.942541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:24.114226 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:24.124357 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:24.125916 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:24.127142 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bdfe4531-2160-4457-b53a-9a5b17935775 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.201598 Jul 31 01:13:24.127786 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 277/540] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:23 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:24.478904 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:24.480027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:24.492335 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:24.493104 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:24.494416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:24.495153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:24.689188 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:24.697725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-61d3c3ea-b93c-460e-b5f7-b565d8d809d5 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.225994 Jul 31 01:13:24.698318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 264/541] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:24 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:25.152275 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:25.153281 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:25.160586 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:25.161036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:25.161831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:25.162272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:25.350364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:25.355394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:25.368694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:25.372133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:25.374379 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-22d15bc5-d47a-4223-9250-3f6b2a6818d3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.226118 Jul 31 01:13:25.375454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 278/542] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:25 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:25.731204 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:25.740474 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:25.759137 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:25.760444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:25.762226 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:25.763016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:25.771219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:25.958830 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:25.969874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9d9614ce-79e6-41c3-9fd7-3db1dc158d91 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1357 microversion: 2.1 time: 0.251830 Jul 31 01:13:25.970486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 265/543] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:25 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1357 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:26.432243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:26.434461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:26.446979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:26.447758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:26.449123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:26.449516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:26.626887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:26.637318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:26.638894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:26.640172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a3b431f2-259a-44c0-963d-29c870cb369d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.217153 Jul 31 01:13:26.640767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 279/544] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:26 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:26.992869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:26.993508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:27.001925 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.002307 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.003112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.003587 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.168629 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:27.184252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-31084175-ffd9-47e1-b684-8c816abc730e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1547 microversion: 2.1 time: 0.195765 Jul 31 01:13:27.184915 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 266/545] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:26 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1547 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:27.207621 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:27.208246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:27.214873 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.215490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.216381 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.216865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.366481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:27.375402 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1f148c0c-6ab4-4795-b9fa-2c009ebc2f44 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1547 microversion: 2.1 time: 0.171492 Jul 31 01:13:27.375922 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 280/546] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:27 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1547 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:27.398247 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:27.403489 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:27.404345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Searching by: {'deleted': False, 'project_id': 'b018ba41816049feaa75e10489ec5281'} {{(pid=117942) get_all /opt/stack/nova/nova/compute/api.py:2964}} Jul 31 01:13:27.411975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.412500 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.413501 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.414660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.519794 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.multi_cell_list [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=117942) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 31 01:13:27.666531 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:27.667163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:27.674223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.674683 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.675515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.675968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.718052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dc9c32ec-af56-454f-8184-151565caa681 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers" status: 200 len: 341 microversion: 2.1 time: 0.324326 Jul 31 01:13:27.719160 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 267/547] 10.222.0.42 () {58 vars in 1125 bytes} [Sun Jul 31 01:13:27 2022] GET /compute/v2.1/servers => generated 341 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:13:27.739898 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:27.742289 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:27.756746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.757948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.759523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:27.760328 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:27.874552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:27.885470 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:27.887083 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:27.888358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-98adfac2-fca3-4c7e-8a1c-cedc3721c421 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.225981 Jul 31 01:13:27.888946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 281/548] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:27 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:27.953091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:27.961768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0caa8870-04f4-4bfd-8b6c-e015b5e70a20 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1547 microversion: 2.1 time: 0.229945 Jul 31 01:13:27.962382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 268/549] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:27 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1547 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:28.914629 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:28.915320 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:28.922575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:28.923031 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:28.923871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:28.924316 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:29.077101 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:29.086650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:29.088296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:29.089605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-95b3ae49-15cf-4ca8-bd2d-16271a76f83e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.179672 Jul 31 01:13:29.090198 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 282/550] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:28 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:30.114006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:30.114762 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:30.121222 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:30.121675 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:30.122471 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:30.122907 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:30.296915 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:30.306862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:30.308498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:30.309899 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-01c98f01-ee8a-479e-8079-3eec2824cd54 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.199893 Jul 31 01:13:30.310548 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 269/551] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:30 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:30.736173 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-19cc0e42-501a-4501-add3-8a7893271334 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "79494aed-a098-435e-80ac-698a6aa289b4"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:30.738697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-19cc0e42-501a-4501-add3-8a7893271334 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:30.746111 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19cc0e42-501a-4501-add3-8a7893271334 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:30.747139 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19cc0e42-501a-4501-add3-8a7893271334 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:30.747408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19cc0e42-501a-4501-add3-8a7893271334 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:30.747838 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19cc0e42-501a-4501-add3-8a7893271334 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:31.335210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:31.335912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:31.342682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:31.343144 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:31.343990 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:31.344440 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:31.519810 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:31.529716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:31.531295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:31.532699 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2beb6ded-ceea-4616-b124-9f98121ee539 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.202193 Jul 31 01:13:31.533347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 270/552] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:31 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:32.557773 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:32.558496 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:32.565106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:32.565564 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:32.566396 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:32.566848 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:32.738813 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:32.748630 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:32.750152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:32.751393 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ec9e5e94-d5a4-4861-9b47-e0a0167d5214 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.197916 Jul 31 01:13:32.752174 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 271/553] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:32 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:33.778883 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:33.780141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:33.790196 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:33.790811 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:33.792083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:33.792618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:33.950038 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:33.961260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:33.962920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:33.964185 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-283e6f24-f2aa-4e5b-acbc-72c5ef9d9074 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.192213 Jul 31 01:13:33.964800 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 272/554] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:33 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:34.993604 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:34.994572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:35.004420 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:35.004974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:35.006292 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:35.007025 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:35.200520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:35.210735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:35.212378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:35.213894 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-99223d71-b7b9-42f3-98d8-c32acd94f189 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.229035 Jul 31 01:13:35.214534 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 273/555] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:34 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:36.238439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:36.239794 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:36.246668 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:36.247128 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:36.247976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:36.248422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:36.429026 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:36.438489 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:36.440153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:36.441388 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0d61e98a-cea7-4d45-9a12-4490709d2d91 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.207015 Jul 31 01:13:36.441984 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 274/556] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:36 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:36.966877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-19cc0e42-501a-4501-add3-8a7893271334 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 6.234549 Jul 31 01:13:36.967798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 283/557] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:13:30 2022] POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments => generated 194 bytes in 6236 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:13:37.469622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:37.470910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:37.473998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:37.481142 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:37.481304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:37.482076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:37.482521 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:37.658992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:37.672968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:37.675200 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:37.677566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-929cbec9-5144-49e7-a17c-c34f868e2c73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.214683 Jul 31 01:13:37.689177 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 275/558] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:37 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:38.712632 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:38.713378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:38.715765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=42 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:38.722625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:38.723085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:38.723943 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:38.724415 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:38.906552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:38.917292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:38.919208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:38.920611 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-06e04733-6a94-4049-abdc-c46a68e0e0bf tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.213817 Jul 31 01:13:38.921329 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 284/559] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:38 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:39.548239 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-b3053f2f-7bbe-4c71-812f-55defd59fe05 req-c3286203-2fbf-4aee-ae7b-25b06f896108 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ef19c2df-372c-4c77-ad6c-ada1cd419a14", "name": "network-vif-plugged", "status": "completed", "tag": "f5779c70-0ae8-4a58-a8ba-25f9117da52e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:39.563955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b3053f2f-7bbe-4c71-812f-55defd59fe05 req-c3286203-2fbf-4aee-ae7b-25b06f896108 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:39.564428 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b3053f2f-7bbe-4c71-812f-55defd59fe05 req-c3286203-2fbf-4aee-ae7b-25b06f896108 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:39.578153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-b3053f2f-7bbe-4c71-812f-55defd59fe05 req-c3286203-2fbf-4aee-ae7b-25b06f896108 service nova] Creating event network-vif-plugged:f5779c70-0ae8-4a58-a8ba-25f9117da52e for instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 on hv-cinder-maste Jul 31 01:13:39.584127 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-b3053f2f-7bbe-4c71-812f-55defd59fe05 req-c3286203-2fbf-4aee-ae7b-25b06f896108 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041780 Jul 31 01:13:39.584907 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 276/560] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:39.945218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:39.945990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:39.953025 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:39.953473 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:39.954266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:39.954705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:40.104279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:40.113724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:40.115534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:40.116747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c7c7f6f3-e26f-4aa4-a346-54fa9180e541 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.175542 Jul 31 01:13:40.117349 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 285/561] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:39 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:41.141377 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:41.142394 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:41.149175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:41.149731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:41.150725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:41.151293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:41.324128 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:41.339820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:41.342917 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:41.345219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4de005fb-e34f-4a95-bd18-d3c2e38c11a2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.208227 Jul 31 01:13:41.346324 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 277/562] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:41 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:42.385729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:42.386539 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:42.395752 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:42.396328 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:42.397323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:42.397856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:42.561146 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:42.571968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:42.574054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:42.575308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f6420f47-b8b6-49b0-a2f5-573e7844964f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1808 microversion: 2.20 time: 0.193725 Jul 31 01:13:42.576476 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 286/563] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:42 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1808 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:13:42.699339 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1414882725", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1706218805", "security_groups": [{"name": "tempest-secgroup-smoke-1479395894"}], "networks": [{"uuid": "8c8c2031-4cdd-4350-8a19-75c824dd85f7"}], "block_device_mapping_v2": [{"uuid": "d0ee94dc-da7a-4b6a-b2d8-9535dfd762e0", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:42.714125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:42.885089 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] validate_networks() for [('8c8c2031-4cdd-4350-8a19-75c824dd85f7', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:13:43.586598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:13:43.587104 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:13:43.587758 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:13:43.588461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:13:43.589228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:13:43.602025 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:43.603412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:43.603538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:43.604352 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:43.606326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:43.607246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:43.612660 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:43.613422 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:43.614811 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:43.615541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:43.641725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Getting quotas for project 5ca9597665cd4068b0a111136d6e0489. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:13:43.650291 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Getting quotas for user cb4bfd7ea6664747a6e0693a2fd11e3b and project 5ca9597665cd4068b0a111136d6e0489. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:13:43.724347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:13:43.726428 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='d0ee94dc-da7a-4b6a-b2d8-9535dfd762e0',volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:13:43.783723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:43.794246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:43.795915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:43.797338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-30af25a8-2d34-40a2-bf8e-67964a7f7fb0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1795 microversion: 2.20 time: 0.200346 Jul 31 01:13:43.797907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 287/564] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:13:43 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1795 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:13:43.918667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-630ef85e-c427-4450-be18-bb28f2c879f9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.223656 Jul 31 01:13:43.919854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 278/565] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:13:42 2022] POST /compute/v2.1/servers => generated 406 bytes in 1226 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:13:43.939372 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-345896fb-3e8a-4903-a4dc-501d654173ca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:43.940061 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-345896fb-3e8a-4903-a4dc-501d654173ca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:44.004222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-345896fb-3e8a-4903-a4dc-501d654173ca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:44.017357 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-345896fb-3e8a-4903-a4dc-501d654173ca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1089 microversion: 2.1 time: 0.082764 Jul 31 01:13:44.019660 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 288/566] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:43 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1089 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:45.045970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:45.046628 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:45.054680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:45.055190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:45.056349 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:45.056792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:45.154185 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:45.162798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-efe885bd-09d0-4a06-b333-18137988390e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1187 microversion: 2.1 time: 0.120867 Jul 31 01:13:45.163439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 279/567] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:45 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1187 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:45.923819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:45.924674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:45.936243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:45.936711 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:45.937634 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:45.938093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:46.173220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:46.185171 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-998ae57c-5490-4ad7-9650-c70b9d3b02e7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1593 microversion: 2.1 time: 0.265217 Jul 31 01:13:46.185970 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 289/568] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:45 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1593 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:46.198774 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:46.199967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:46.209230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:46.210099 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:46.211237 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:46.212006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:46.308299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:46.321897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ead8e1a6-aa14-455b-9b3e-c1a1096a5c5f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1205 microversion: 2.1 time: 0.127858 Jul 31 01:13:46.335402 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 280/569] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:46 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1205 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:47.362204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:47.363007 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:47.375103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:47.376419 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:47.378280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:47.379103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:47.667655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:47.684695 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-372b1e75-2c12-4699-877f-0cef05d79af9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.327567 Jul 31 01:13:47.686482 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 290/570] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:47 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:47.775897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-93e334c5-6c11-41cf-aeb2-a0d547c3f6ea req-4b8ae79a-2472-42fe-bf70-ff0c47ee362e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69cf3658-0cb6-48b8-b152-72b915385fbb", "tag": "38f41703-e2b2-4f28-9dc2-9118c346cfba"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:47.797662 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-93e334c5-6c11-41cf-aeb2-a0d547c3f6ea req-4b8ae79a-2472-42fe-bf70-ff0c47ee362e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:47.798808 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-93e334c5-6c11-41cf-aeb2-a0d547c3f6ea req-4b8ae79a-2472-42fe-bf70-ff0c47ee362e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:47.823441 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-93e334c5-6c11-41cf-aeb2-a0d547c3f6ea req-4b8ae79a-2472-42fe-bf70-ff0c47ee362e service nova] Creating event network-changed:38f41703-e2b2-4f28-9dc2-9118c346cfba for instance 69cf3658-0cb6-48b8-b152-72b915385fbb on hv-cinder-maste Jul 31 01:13:47.835918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-93e334c5-6c11-41cf-aeb2-a0d547c3f6ea req-4b8ae79a-2472-42fe-bf70-ff0c47ee362e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071150 Jul 31 01:13:47.837194 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 281/571] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:48.714185 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:48.715108 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:48.725287 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:48.725840 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:48.727133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:48.727931 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:48.967484 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:48.983981 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3b392d0a-c5ec-49e6-ae24-261b870c745c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.273737 Jul 31 01:13:48.985443 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 291/572] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:48 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:49.154533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-b7d5f0e3-bd63-4e8d-ad8c-e9e176a80250 req-5fcc04e3-0cae-4343-a0d8-3d83320d118a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7a3b72f2-b29c-4115-b742-1fa4a86e06b6", "tag": "f704de2f-7bdb-4ef2-80b7-29cb6b9472eb"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:49.180786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b7d5f0e3-bd63-4e8d-ad8c-e9e176a80250 req-5fcc04e3-0cae-4343-a0d8-3d83320d118a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:49.182136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b7d5f0e3-bd63-4e8d-ad8c-e9e176a80250 req-5fcc04e3-0cae-4343-a0d8-3d83320d118a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:49.199123 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-b7d5f0e3-bd63-4e8d-ad8c-e9e176a80250 req-5fcc04e3-0cae-4343-a0d8-3d83320d118a service nova] Creating event network-changed:f704de2f-7bdb-4ef2-80b7-29cb6b9472eb for instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 on hv-cinder-maste Jul 31 01:13:49.205312 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-b7d5f0e3-bd63-4e8d-ad8c-e9e176a80250 req-5fcc04e3-0cae-4343-a0d8-3d83320d118a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.143990 Jul 31 01:13:49.206311 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 282/573] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:50.022822 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:50.025278 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:50.044625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:50.045549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:50.047996 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:50.048725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:50.403336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:50.421369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d2febe83-e22e-4388-80f1-263df94e2f1a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.404031 Jul 31 01:13:50.436686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 292/574] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:50 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:50.893304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-386677154"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:50.894017 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:50.901512 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:50.902119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:50.903009 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:50.903581 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:51.067657 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.network.security_group_api [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Adding security group 6f0c7be0-f500-456b-b9a9-a904ad09a297 to port f704de2f-7bdb-4ef2-80b7-29cb6b9472eb Jul 31 01:13:51.459324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:51.460014 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:51.468835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:51.469396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:51.476962 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:51.477629 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:51.680747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:51.701596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-aeca837b-8b25-4f58-b3ce-c54c76fbc977 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.247350 Jul 31 01:13:51.702232 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 293/575] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:51 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:51.747891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cd24be36-c6df-4dbe-b152-3984afbf4a3e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/action" status: 202 len: 0 microversion: 2.1 time: 0.859052 Jul 31 01:13:51.748587 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 283/576] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:13:50 2022] POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/action => generated 0 bytes in 861 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:13:51.756745 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-9e100d83-ebc9-4479-9e10-faf8a72f836c req-15d69ff1-9c46-48d0-846c-4560def23199 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7a3b72f2-b29c-4115-b742-1fa4a86e06b6", "tag": "f704de2f-7bdb-4ef2-80b7-29cb6b9472eb"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:51.757319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:51.757952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:51.764849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:51.765562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:51.766372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:51.766492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-9e100d83-ebc9-4479-9e10-faf8a72f836c req-15d69ff1-9c46-48d0-846c-4560def23199 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:51.766832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:51.766976 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-9e100d83-ebc9-4479-9e10-faf8a72f836c req-15d69ff1-9c46-48d0-846c-4560def23199 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:51.779904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-9e100d83-ebc9-4479-9e10-faf8a72f836c req-15d69ff1-9c46-48d0-846c-4560def23199 service nova] Creating event network-changed:f704de2f-7bdb-4ef2-80b7-29cb6b9472eb for instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 on hv-cinder-maste Jul 31 01:13:51.785368 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-9e100d83-ebc9-4479-9e10-faf8a72f836c req-15d69ff1-9c46-48d0-846c-4560def23199 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033394 Jul 31 01:13:51.786303 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 294/577] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:51.790524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:51.987716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:51.997878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-00bf2d1c-409c-4e9b-8d23-8a8bbec5e88c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1757 microversion: 2.1 time: 0.245048 Jul 31 01:13:51.998473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 284/578] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:51 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1757 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:52.725810 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:52.726483 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:52.734061 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:52.734523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:52.735315 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:52.735797 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:52.886990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:52.890643 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:52.896261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8f6982f2-a69f-4bf4-8c2b-47ea98795b57 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.177140 Jul 31 01:13:52.896846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 295/579] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:52 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:53.917533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:53.918423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:53.920741 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:53.921541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=6 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:53.929147 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:53.929600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:53.930656 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:53.931088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:54.079640 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:54.087930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-119a7fde-8b57-4db9-b7d2-a404e51f2b3b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.174402 Jul 31 01:13:54.088517 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 285/580] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:53 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:13:55.109988 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:55.110605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:55.114264 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:13:55.120452 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:55.120904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:55.121689 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:55.122130 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:55.269398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:55.278305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-efce9c15-0531-459c-a101-3d2d98703cab tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.172097 Jul 31 01:13:55.279091 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 296/581] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:55 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:56.301063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:56.301921 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:56.309072 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:56.309517 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:56.310293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:56.310729 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:56.459849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:56.469013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1a6b3e17-16a3-4e36-997d-11c800a8e246 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.172287 Jul 31 01:13:56.469615 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 286/582] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:56 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:57.494451 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:57.495378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:57.502967 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:57.503464 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:57.504902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:57.505343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:57.701391 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:57.714525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a40eeb6c-e0c4-495a-ae8d-0f2f069a1715 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.224252 Jul 31 01:13:57.715807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 297/583] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:57 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:58.740039 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:58.740629 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:58.747468 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:58.748033 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:58.748875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:58.749369 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:58.910044 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:13:58.919050 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1ad13220-6026-4e34-a6b2-34181232d05b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.182957 Jul 31 01:13:58.919772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 287/584] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:58 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:13:59.789475 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-2ed924b6-a1fe-4807-959b-2a7b77d35b3f req-6dc0688f-8423-4025-affe-20b71625d1d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cf3658-0cb6-48b8-b152-72b915385fbb", "name": "network-vif-plugged", "status": "completed", "tag": "38f41703-e2b2-4f28-9dc2-9118c346cfba"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:13:59.806742 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-2ed924b6-a1fe-4807-959b-2a7b77d35b3f req-6dc0688f-8423-4025-affe-20b71625d1d3 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:59.807494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-2ed924b6-a1fe-4807-959b-2a7b77d35b3f req-6dc0688f-8423-4025-affe-20b71625d1d3 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:59.822965 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-2ed924b6-a1fe-4807-959b-2a7b77d35b3f req-6dc0688f-8423-4025-affe-20b71625d1d3 service nova] Creating event network-vif-plugged:38f41703-e2b2-4f28-9dc2-9118c346cfba for instance 69cf3658-0cb6-48b8-b152-72b915385fbb on hv-cinder-maste Jul 31 01:13:59.829239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-2ed924b6-a1fe-4807-959b-2a7b77d35b3f req-6dc0688f-8423-4025-affe-20b71625d1d3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047543 Jul 31 01:13:59.830184 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 298/585] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:13:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:13:59.941809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:13:59.942417 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:13:59.949631 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:59.950066 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:13:59.950909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:13:59.951346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:00.135388 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:00.148084 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c9c9540d-b016-45f9-aa67-44d3a4d20e0f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.210027 Jul 31 01:14:00.148917 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 288/586] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:13:59 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:01.182481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:01.183708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:01.192536 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:01.192988 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:01.193803 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:01.194254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:01.355221 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:01.363484 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c47e8a88-1002-428c-aa8c-a802cf05a199 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1261 microversion: 2.1 time: 0.186687 Jul 31 01:14:01.364093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 299/587] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:01 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1261 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:01.910412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-818d1280-1206-4727-a2f8-24381197f2fd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:01.911160 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-818d1280-1206-4727-a2f8-24381197f2fd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:01.918154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-818d1280-1206-4727-a2f8-24381197f2fd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:01.918755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-818d1280-1206-4727-a2f8-24381197f2fd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:01.919597 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-818d1280-1206-4727-a2f8-24381197f2fd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:01.920079 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-818d1280-1206-4727-a2f8-24381197f2fd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:01.948023 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-818d1280-1206-4727-a2f8-24381197f2fd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf/ips" status: 200 len: 150 microversion: 2.1 time: 0.158602 Jul 31 01:14:01.948875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 289/588] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:14:01 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf/ips => generated 150 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:01.962038 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "44ee31f1-a5f4-4b51-b90c-15f7f7430a97"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:01.964679 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:01.971236 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:01.971730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:01.972564 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:01.973057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.072861 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Invalid volume: volume 44ee31f1-a5f4-4b51-b90c-15f7f7430a97 is already attached to instances: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: POST /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf/os-volume_attachments HTTP/1.1 Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 74 Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:14:02.074720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {"volumeAttachment": {"volumeId": "44ee31f1-a5f4-4b51-b90c-15f7f7430a97"}} Jul 31 01:14:02.077230 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Invalid volume: volume 44ee31f1-a5f4-4b51-b90c-15f7f7430a97 is already attached to instances: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'server_id': '2b53247d-da0a-46f9-aa9e-f4d09342e5bf', 'body': {'volumeAttachment': {'volumeId': '44ee31f1-a5f4-4b51-b90c-15f7f7430a97'}}} Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5066, in attach_volume Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4906, in _check_volume_already_attached Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 44ee31f1-a5f4-4b51-b90c-15f7f7430a97 is already attached to instances: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:02.082950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 44ee31f1-a5f4-4b51-b90c-15f7f7430a97 is already attached to instances: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:02.085047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 44ee31f1-a5f4-4b51-b90c-15f7f7430a97 is already attached to instances: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 400 to user: Invalid volume: volume 44ee31f1-a5f4-4b51-b90c-15f7f7430a97 is already attached to instances: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0909aa6b-7e18-4d05-9acf-6b403afa77cb tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.119033 Jul 31 01:14:02.087160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 300/589] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:14:01 2022] POST /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf/os-volume_attachments => generated 174 bytes in 122 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 31 01:14:02.090284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:02.090920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:02.098661 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.099273 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.100279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.100772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.129724 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:14:02.220168 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e9adfa0d-e2a1-47e7-8c62-07f473a7f344 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 204 len: 0 microversion: 2.1 time: 0.135514 Jul 31 01:14:02.220775 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 290/590] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:14:02 2022] DELETE /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:14:02.229370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:02.230218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:02.239594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.240224 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.241112 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.241643 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.391970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:02.393452 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:02.405045 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.405641 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.407844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.408774 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.416664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:02.433966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-909ca744-7040-440f-98f7-189ce04d2a64 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1682 microversion: 2.1 time: 0.207851 Jul 31 01:14:02.435780 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 301/591] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:02 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1682 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:02.597612 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:02.607750 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-eb45dbe2-8567-48ad-95e8-be774ecc908e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1445 microversion: 2.1 time: 0.224875 Jul 31 01:14:02.608549 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 291/592] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:02 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1445 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:02.631821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:02.632537 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:02.639286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.639818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.640688 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:02.641150 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:02.794536 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:02.804884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2d6131df-2452-4c49-88d7-746acd90e7da tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1445 microversion: 2.1 time: 0.177362 Jul 31 01:14:02.805906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 302/593] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:02 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1445 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:03.461319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:03.461890 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:03.468638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:03.469171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:03.470029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:03.470529 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:03.632476 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:03.642636 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6a5532c4-3239-4711-ae5a-c497e807697a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1682 microversion: 2.1 time: 0.186049 Jul 31 01:14:03.644687 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 292/594] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:03 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1682 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:04.673912 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:04.675029 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:04.692209 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:04.692852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:04.694257 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:04.695052 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:05.076928 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:05.093458 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-656d3787-2afa-4799-a210-3311962d9380 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1682 microversion: 2.1 time: 0.426537 Jul 31 01:14:05.094737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 303/595] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:04 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1682 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:05.384368 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-ef626a94-620d-4d75-9462-339af1b2c534 req-2869e314-a647-46c5-bf98-0cb8816bf3db service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69cf3658-0cb6-48b8-b152-72b915385fbb", "tag": "38f41703-e2b2-4f28-9dc2-9118c346cfba"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:05.403624 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ef626a94-620d-4d75-9462-339af1b2c534 req-2869e314-a647-46c5-bf98-0cb8816bf3db service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:05.404556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ef626a94-620d-4d75-9462-339af1b2c534 req-2869e314-a647-46c5-bf98-0cb8816bf3db service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:05.430536 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-ef626a94-620d-4d75-9462-339af1b2c534 req-2869e314-a647-46c5-bf98-0cb8816bf3db service nova] Creating event network-changed:38f41703-e2b2-4f28-9dc2-9118c346cfba for instance 69cf3658-0cb6-48b8-b152-72b915385fbb on hv-cinder-maste Jul 31 01:14:05.444128 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-ef626a94-620d-4d75-9462-339af1b2c534 req-2869e314-a647-46c5-bf98-0cb8816bf3db service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066469 Jul 31 01:14:05.447752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 293/596] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:14:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:06.101654 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-ab080cea-b8f2-4644-9e0f-a9b7ab0230ee req-e22bf402-9a7a-4239-b016-5e17431f7d02 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b53247d-da0a-46f9-aa9e-f4d09342e5bf", "name": "network-vif-deleted", "tag": "b48b5519-311b-4ddf-9138-fa82f7ac77af"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:06.123992 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-ab080cea-b8f2-4644-9e0f-a9b7ab0230ee req-e22bf402-9a7a-4239-b016-5e17431f7d02 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:06.125165 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-ab080cea-b8f2-4644-9e0f-a9b7ab0230ee req-e22bf402-9a7a-4239-b016-5e17431f7d02 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:06.132782 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:06.135554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:06.148678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:06.150329 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:06.152372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:06.153585 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:06.167879 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-ab080cea-b8f2-4644-9e0f-a9b7ab0230ee req-e22bf402-9a7a-4239-b016-5e17431f7d02 service nova] Creating event network-vif-deleted:b48b5519-311b-4ddf-9138-fa82f7ac77af for instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf on hv-cinder-maste Jul 31 01:14:06.184327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-ab080cea-b8f2-4644-9e0f-a9b7ab0230ee req-e22bf402-9a7a-4239-b016-5e17431f7d02 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.199753 Jul 31 01:14:06.185988 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 304/597] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:14:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:14:06.333748 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:06.344428 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-50693ab3-6dd5-44ed-9579-aa980d5efc8b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 200 len: 1615 microversion: 2.1 time: 0.222818 Jul 31 01:14:06.345755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 294/598] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:06 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 1615 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:07.387776 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:07.390371 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:07.409823 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:07.410656 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:07.411800 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:07.412490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:07.432152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf HTTP/1.1 Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:14:07.434655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:14:07.436121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '2b53247d-da0a-46f9-aa9e-f4d09342e5bf'} Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:14:07.436846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:14:07.437985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:14:07.441150 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:14:07.442610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7d7b3bce-21fb-4713-87cf-6b9ac6b80983 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 404 len: 111 microversion: 2.1 time: 0.065603 Jul 31 01:14:07.447565 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 305/599] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:07 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:14:07.459596 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dd03659a-a39c-430d-8cad-f668067b0acc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '.version_select at 0x7f3e8009fdc0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:07.510107 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dd03659a-a39c-430d-8cad-f668067b0acc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1892841760" status: 202 len: 0 microversion: 2.1 time: 0.060639 Jul 31 01:14:07.512843 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 295/600] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Jul 31 01:14:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1892841760 => generated 0 bytes in 64 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 31 01:14:08.483006 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:08.483918 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:08.497401 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:08.498547 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:08.500136 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:08.500966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:08.725345 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lazy-loading 'services' on Instance uuid 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:14:08.892266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d094e9ae-35d0-455b-9e88-eff49a9d2ed5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments/44ee31f1-a5f4-4b51-b90c-15f7f7430a97" status: 202 len: 0 microversion: 2.1 time: 0.417256 Jul 31 01:14:08.892889 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 306/601] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:14:08 2022] DELETE /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments/44ee31f1-a5f4-4b51-b90c-15f7f7430a97 => generated 0 bytes in 419 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:14:08.907279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-608bd5e5-eab9-4e10-9c26-4f4ff84394c3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:08.912083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-608bd5e5-eab9-4e10-9c26-4f4ff84394c3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:08.923688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608bd5e5-eab9-4e10-9c26-4f4ff84394c3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:08.924353 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608bd5e5-eab9-4e10-9c26-4f4ff84394c3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:08.925825 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608bd5e5-eab9-4e10-9c26-4f4ff84394c3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:08.926347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608bd5e5-eab9-4e10-9c26-4f4ff84394c3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:08.965034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-608bd5e5-eab9-4e10-9c26-4f4ff84394c3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066255 Jul 31 01:14:08.965639 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 296/602] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:14:08 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:09.985280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a2bc484a-3597-42c9-a1d6-c500a786db19 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:09.989190 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a2bc484a-3597-42c9-a1d6-c500a786db19 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:09.993850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=48 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:10.004375 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2bc484a-3597-42c9-a1d6-c500a786db19 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:10.004918 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2bc484a-3597-42c9-a1d6-c500a786db19 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:10.005817 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2bc484a-3597-42c9-a1d6-c500a786db19 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:10.006504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a2bc484a-3597-42c9-a1d6-c500a786db19 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:10.045727 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a2bc484a-3597-42c9-a1d6-c500a786db19 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069599 Jul 31 01:14:10.046308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 307/603] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:14:09 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:11.059128 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-31a8f8e2-711d-43e4-9aa0-1570c8c45512 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:11.061481 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-31a8f8e2-711d-43e4-9aa0-1570c8c45512 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:11.068811 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31a8f8e2-711d-43e4-9aa0-1570c8c45512 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:11.069413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31a8f8e2-711d-43e4-9aa0-1570c8c45512 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:11.070359 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31a8f8e2-711d-43e4-9aa0-1570c8c45512 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:11.070875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-31a8f8e2-711d-43e4-9aa0-1570c8c45512 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:11.104699 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-31a8f8e2-711d-43e4-9aa0-1570c8c45512 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049890 Jul 31 01:14:11.105192 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 297/604] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:14:11 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:14:11.183242 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:11.183878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:11.191262 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:11.191755 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:11.192545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:11.192983 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:11.226427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:14:11.320316 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0c108a41-13c8-4723-b2ff-dac7f07207a5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 204 len: 0 microversion: 2.1 time: 0.140633 Jul 31 01:14:11.321290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 308/605] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:14:11 2022] DELETE /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:14:11.334761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:11.335919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:11.347040 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:11.347880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:11.349265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:11.350055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:11.505041 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:11.513178 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-deee26b7-c124-4f05-a8cc-d885256b89ef tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1683 microversion: 2.1 time: 0.185806 Jul 31 01:14:11.513855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 298/606] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:11 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1683 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:12.538563 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:12.539449 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:12.546252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:12.546694 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:12.547516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:12.547946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:12.704553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:12.714033 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0c8805b5-dba9-4f69-a3c6-dc06a694226e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1683 microversion: 2.1 time: 0.179752 Jul 31 01:14:12.714639 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 309/607] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:12 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1683 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:13.738058 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:13.738800 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:13.741034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=3,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:13.747124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:13.747618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:13.748415 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:13.748854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:13.843285 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:13.857506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2e95bfd5-0ebe-40f1-b3e1-c4675f315492 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 200 len: 1615 microversion: 2.1 time: 0.124559 Jul 31 01:14:13.858218 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 299/608] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:13 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 1615 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:13.913595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-7b66d115-e353-451e-a06d-3b38c5a02f50 req-186b0e3e-7041-47cf-a181-b5b4adf1041e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ffec4de-cf74-4e4f-acc9-ea172ae3e171", "name": "network-vif-deleted", "tag": "e754ffa9-197f-4f94-a875-08f75b6e8593"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:13.935993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-7b66d115-e353-451e-a06d-3b38c5a02f50 req-186b0e3e-7041-47cf-a181-b5b4adf1041e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:13.936586 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-7b66d115-e353-451e-a06d-3b38c5a02f50 req-186b0e3e-7041-47cf-a181-b5b4adf1041e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:13.972215 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-7b66d115-e353-451e-a06d-3b38c5a02f50 req-186b0e3e-7041-47cf-a181-b5b4adf1041e service nova] Creating event network-vif-deleted:e754ffa9-197f-4f94-a875-08f75b6e8593 for instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 on hv-cinder-maste Jul 31 01:14:13.983217 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-7b66d115-e353-451e-a06d-3b38c5a02f50 req-186b0e3e-7041-47cf-a181-b5b4adf1041e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075094 Jul 31 01:14:13.984359 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 310/609] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:14:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:14.898770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:14.900243 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:14.906888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:14.907337 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:14.908166 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:14.908604 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:14.923856 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 HTTP/1.1 Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:14:14.926071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:14:14.927820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '7ffec4de-cf74-4e4f-acc9-ea172ae3e171'} Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:14:14.929214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:14:14.931719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:14:14.934266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:14:14.934266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fe3ab53a-ebdf-442b-8dda-c102d69be43d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 404 len: 111 microversion: 2.1 time: 0.032944 Jul 31 01:14:14.934266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 300/610] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:14 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:14:14.944222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-37d4e971-6274-4465-b352-867a922802cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '.version_select at 0x7f3e7b7cf9d0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:14.969261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-37d4e971-6274-4465-b352-867a922802cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1298245109" status: 202 len: 0 microversion: 2.1 time: 0.032998 Jul 31 01:14:14.969860 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 311/611] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Jul 31 01:14:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1298245109 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:14:15.872290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f4999fe4-b27f-4db6-8560-b79bc10af9ed tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: .version_select at 0x7f3e7b70b790>, body: {"keypair": {"name": "tempest-keypair-879799212", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKXiBDu7ntwj9oXn5oXXp3EraEb8wj3tHkCOZUOVcFxk+JSFqq4CpAc1xniCq6d2y1nOBTkfDjZre/WgqJP9eWX3SZLmxZFtE31Zj2c4aYSYVJ16whIWrxlNhz7VX3xeAg=="}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:15.894816 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f4999fe4-b27f-4db6-8560-b79bc10af9ed tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:14:15.900512 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-f4999fe4-b27f-4db6-8560-b79bc10af9ed tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:14:15.927343 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f4999fe4-b27f-4db6-8560-b79bc10af9ed tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.059301 Jul 31 01:14:15.928009 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 301/612] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:14:15 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:14:17.415217 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-973077789", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b"}], "security_groups": [{"name": "tempest-securitygroup--147097705"}], "key_name": "tempest-keypair-879799212", "user_data": "}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:17.587300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] validate_networks() for [('3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:14:18.183093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:14:18.183732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:14:18.184320 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:14:18.184883 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:14:18.185556 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:14:18.197662 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:18.198137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:18.198982 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:18.199457 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:18.220292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:14:18.226489 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:14:18.241072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:14:18.243655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:14:18.300369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7685c506-bd10-442e-bcb0-d4945ea7fe03 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.890838 Jul 31 01:14:18.301478 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 312/613] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:14:17 2022] POST /compute/v2.1/servers => generated 401 bytes in 893 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 31 01:14:18.318376 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-28801db9-8331-40be-b1b2-12d57e0dfa43 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:18.319762 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-28801db9-8331-40be-b1b2-12d57e0dfa43 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:18.400284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-28801db9-8331-40be-b1b2-12d57e0dfa43 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:18.411951 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-28801db9-8331-40be-b1b2-12d57e0dfa43 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1240 microversion: 2.1 time: 0.100248 Jul 31 01:14:18.413040 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 302/614] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:18 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1240 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:19.453562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:19.454647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:19.463020 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:19.463594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:19.464436 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:19.464888 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:19.551907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:19.555582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f95f5ecf-a25a-4442-a166-248bb671bc23 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1292 microversion: 2.1 time: 0.110437 Jul 31 01:14:19.556973 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 313/615] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:19 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1292 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:20.594551 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:20.595724 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:20.617212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:20.618064 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:20.618927 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:20.619428 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:20.820395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:20.829122 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f9ec5c9b-bc26-4bb3-af7b-6f61799c0d14 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.243028 Jul 31 01:14:20.829732 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 303/616] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:20 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:21.358402 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-2fa77f3b-c1a2-4a04-8359-25460c74f647 req-9268471d-c6de-4436-b14d-0dfba17b44be service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1cf1b41-8f07-4466-b084-79a321360da3", "tag": "59fd75bb-d9d0-4f3d-883e-d97aed7927ed"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:21.379533 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-2fa77f3b-c1a2-4a04-8359-25460c74f647 req-9268471d-c6de-4436-b14d-0dfba17b44be service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:21.380459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-2fa77f3b-c1a2-4a04-8359-25460c74f647 req-9268471d-c6de-4436-b14d-0dfba17b44be service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:21.403693 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-2fa77f3b-c1a2-4a04-8359-25460c74f647 req-9268471d-c6de-4436-b14d-0dfba17b44be service nova] Creating event network-changed:59fd75bb-d9d0-4f3d-883e-d97aed7927ed for instance d1cf1b41-8f07-4466-b084-79a321360da3 on hv-cinder-maste Jul 31 01:14:21.412576 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-2fa77f3b-c1a2-4a04-8359-25460c74f647 req-9268471d-c6de-4436-b14d-0dfba17b44be service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063012 Jul 31 01:14:21.414619 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 314/617] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:14:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:21.855336 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:21.856203 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:21.873498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:21.874393 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:21.875991 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:21.876465 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:21.879986 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:22.075478 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:22.085698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0a683e72-96a1-4f9f-b2b0-f1186193cc9d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.234796 Jul 31 01:14:22.086506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 304/618] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:21 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:23.111566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:23.112201 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:23.119502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:23.119792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:23.120588 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:23.121044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:23.126296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:23.286299 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:23.300335 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f81380d6-764b-4581-a26a-1613edccca81 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.193589 Jul 31 01:14:23.301274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 315/619] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:23 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:24.323394 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:24.324088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:24.331347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:24.331761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:24.332643 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:24.333088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:24.488647 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:24.496815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0eb6d2a9-77dc-4833-bfa5-ef9fcb6f3363 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.178294 Jul 31 01:14:24.497415 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 305/620] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:24 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:25.520415 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:25.521242 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:25.529929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:25.530378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:25.531168 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:25.531642 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:25.718254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:25.730156 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-79c477b3-a7c9-4158-bba1-b582a8ecd4de tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.214115 Jul 31 01:14:25.731179 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 316/621] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:25 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:26.755198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:26.756323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:26.765095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:26.765747 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:26.766855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:26.767478 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:26.964614 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:26.973938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-39c990fd-70a4-4e63-b80e-b9fddfee70f2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.223578 Jul 31 01:14:26.974550 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 306/622] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:26 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:27.996000 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:27.996651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:28.004503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:28.004972 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:28.005790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:28.006263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:28.158068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:28.166386 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8ee7dd5f-e109-427e-8898-57fb2bdb8a01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.174253 Jul 31 01:14:28.167017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 317/623] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:27 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:28.378763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-560cbd62-68c0-4085-9f38-ab6b302764ad req-9183a02f-b3df-490a-9c29-8c5918daea4f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1cf1b41-8f07-4466-b084-79a321360da3", "name": "network-vif-plugged", "status": "completed", "tag": "59fd75bb-d9d0-4f3d-883e-d97aed7927ed"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:28.394376 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-560cbd62-68c0-4085-9f38-ab6b302764ad req-9183a02f-b3df-490a-9c29-8c5918daea4f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:28.395695 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-560cbd62-68c0-4085-9f38-ab6b302764ad req-9183a02f-b3df-490a-9c29-8c5918daea4f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:28.413959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=6 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:28.416624 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-560cbd62-68c0-4085-9f38-ab6b302764ad req-9183a02f-b3df-490a-9c29-8c5918daea4f service nova] Creating event network-vif-plugged:59fd75bb-d9d0-4f3d-883e-d97aed7927ed for instance d1cf1b41-8f07-4466-b084-79a321360da3 on hv-cinder-maste Jul 31 01:14:28.422301 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-560cbd62-68c0-4085-9f38-ab6b302764ad req-9183a02f-b3df-490a-9c29-8c5918daea4f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.127831 Jul 31 01:14:28.422931 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 307/624] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:14:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:29.192008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:29.192713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:29.194629 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:29.195319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:29.202374 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:29.202830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:29.203660 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:29.204101 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:29.360361 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:29.368859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0cdea167-9031-4d84-96b1-66cad40c340d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1365 microversion: 2.1 time: 0.182350 Jul 31 01:14:29.369580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 318/625] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:29 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1365 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:30.414934 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:30.416390 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:30.426201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:30.426893 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:30.428070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:30.428744 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:30.590612 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:30.605114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-21418571-416c-4152-bad4-4d377a1711b5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1555 microversion: 2.1 time: 0.200335 Jul 31 01:14:30.606236 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 308/626] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:30 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1555 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:30.645691 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4cae0fd5-a67c-4ecb-8d10-f99b04406e15 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:30.646498 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4cae0fd5-a67c-4ecb-8d10-f99b04406e15 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:30.659035 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4cae0fd5-a67c-4ecb-8d10-f99b04406e15 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:30.659907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4cae0fd5-a67c-4ecb-8d10-f99b04406e15 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:30.661369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4cae0fd5-a67c-4ecb-8d10-f99b04406e15 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:30.662162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4cae0fd5-a67c-4ecb-8d10-f99b04406e15 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:30.778606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4cae0fd5-a67c-4ecb-8d10-f99b04406e15 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.137729 Jul 31 01:14:30.779906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 319/627] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:14:30 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-interface => generated 284 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:32.232804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-71852b83-aa1e-493f-89e8-60e940e6d730 req-b49dd9ad-6b60-4d06-8dda-6dac59e10aed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1cf1b41-8f07-4466-b084-79a321360da3", "tag": "59fd75bb-d9d0-4f3d-883e-d97aed7927ed"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:32.253448 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-71852b83-aa1e-493f-89e8-60e940e6d730 req-b49dd9ad-6b60-4d06-8dda-6dac59e10aed service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:32.254315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-71852b83-aa1e-493f-89e8-60e940e6d730 req-b49dd9ad-6b60-4d06-8dda-6dac59e10aed service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:32.276999 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-71852b83-aa1e-493f-89e8-60e940e6d730 req-b49dd9ad-6b60-4d06-8dda-6dac59e10aed service nova] Creating event network-changed:59fd75bb-d9d0-4f3d-883e-d97aed7927ed for instance d1cf1b41-8f07-4466-b084-79a321360da3 on hv-cinder-maste Jul 31 01:14:32.286960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-71852b83-aa1e-493f-89e8-60e940e6d730 req-b49dd9ad-6b60-4d06-8dda-6dac59e10aed service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061644 Jul 31 01:14:32.288205 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 309/628] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:14:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:36.028219 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f2b399fa-7071-4706-8e41-3cfa2d8c5800 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0815bc2c-8dad-4c45-93ce-5e1618fad030"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:36.030672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f2b399fa-7071-4706-8e41-3cfa2d8c5800 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:36.038554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f2b399fa-7071-4706-8e41-3cfa2d8c5800 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:36.039158 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f2b399fa-7071-4706-8e41-3cfa2d8c5800 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:36.040290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f2b399fa-7071-4706-8e41-3cfa2d8c5800 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:36.040829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f2b399fa-7071-4706-8e41-3cfa2d8c5800 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:36.397883 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f2b399fa-7071-4706-8e41-3cfa2d8c5800 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.374315 Jul 31 01:14:36.398713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 320/629] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:14:36 2022] POST /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments => generated 194 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:14:44.029974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:14:44.032401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:44.034755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=32,nova_api:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:44.035466 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=27 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:44.042529 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:44.042988 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:44.043837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:44.044288 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:44.072807 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lazy-loading 'flavor' on Instance uuid 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:14:44.224777 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8bfc6806-308c-484f-9398-0453cef9add4 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/action" status: 200 len: 32 microversion: 2.1 time: 0.199288 Jul 31 01:14:44.225763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 310/630] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:14:44 2022] POST /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/action => generated 32 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:14:44.243178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:44.244564 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:44.248252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:44.259676 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:44.260508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:44.261932 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:44.262609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:44.541787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:44.555895 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b59e2e5e-8083-40a3-a6b7-ea858e828bf5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.321280 Jul 31 01:14:44.556241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 321/631] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:44 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:45.579213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:45.579957 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:45.586864 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:45.587447 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:45.588263 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:45.588706 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:45.753976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:45.763594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4361ae73-fc59-4617-a4ba-7819e50e4a13 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.188576 Jul 31 01:14:45.764254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 311/632] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:45 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:46.786451 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:46.787148 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:46.794374 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:46.794819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:46.795651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:46.796097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:46.950557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:46.959481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1071578c-4181-42e8-92c3-e67724092c87 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.176996 Jul 31 01:14:46.960085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 322/633] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:46 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:47.982537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:47.983221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:47.990219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:47.990712 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:47.991571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:47.992029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:48.162799 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:48.174157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7ee34b2f-5894-473a-ad48-9820b57ac068 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.195818 Jul 31 01:14:48.174774 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 312/634] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:47 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:49.200093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:49.200818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:49.208238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:49.208710 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:49.209501 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:49.209941 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:49.372997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:49.383440 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0decab8d-6390-4409-8f57-6e495befce81 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.187428 Jul 31 01:14:49.384066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 323/635] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:49 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:50.406924 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:50.408170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:50.415802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:50.416375 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:50.417189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:50.417634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:50.588586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:50.598010 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-948c1930-02b4-4a50-81e8-a2a3b733b0ef tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.195824 Jul 31 01:14:50.598621 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 313/636] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:50 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:51.620493 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:51.621109 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:51.627706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:51.628160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:51.628952 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:51.629394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:51.779560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:51.789373 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-67bff956-6da7-4028-85f7-be3968bb4464 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.172868 Jul 31 01:14:51.789970 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 324/637] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:51 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:52.815691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:52.816769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:52.830353 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:52.831162 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:52.832644 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:52.833436 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:53.002560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:53.019586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-955eeb19-4ad5-4af2-ba61-7bbbbefd4128 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.209281 Jul 31 01:14:53.020394 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 314/638] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:52 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:54.042638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:54.043561 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:54.051927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:54.052536 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:54.053438 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:54.054020 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:54.060478 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=62,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:14:54.213021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:54.222330 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b28f674a-7ec0-448a-abdd-b09a5eca93a5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.184157 Jul 31 01:14:54.222977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 325/639] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:54 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:14:55.244512 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:55.245109 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:55.252178 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:55.252775 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:55.253810 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:55.254377 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:55.408822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:55.417844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-80202345-610e-457b-a963-693a9c48f380 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.177395 Jul 31 01:14:55.418450 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 315/640] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:55 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:56.440918 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:56.442005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:56.450210 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:56.450708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:56.451718 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:56.452221 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:56.608253 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:56.617135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-748b1489-e4ca-4b8e-9dca-62662c3140c0 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.180329 Jul 31 01:14:56.617729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 326/641] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:56 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:57.639084 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:57.640015 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:57.646486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:57.646944 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:57.647772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:57.648218 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:57.803145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:57.812264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9027c831-f4d6-44ee-8ce4-6d3aa30a24d7 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.177143 Jul 31 01:14:57.812922 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 316/642] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:57 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:14:58.835208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:14:58.835906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:14:58.843472 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:58.844130 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:58.845712 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:14:58.846247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:14:59.000392 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:14:59.010187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9eb7d437-9bb2-4711-931d-a0903667154b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.178756 Jul 31 01:14:59.010990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 327/643] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:14:58 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:00.033323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:00.034023 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:00.040635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.041145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.041973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.042455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.191219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:00.199867 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1e0090c8-4bc8-4e53-a1e4-867416dfa575 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.170886 Jul 31 01:15:00.200725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 317/644] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:00 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:00.335526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-709dc128-d4ea-4277-9abe-a052475ae78e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:00.336424 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-709dc128-d4ea-4277-9abe-a052475ae78e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:00.343371 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-709dc128-d4ea-4277-9abe-a052475ae78e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.343845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-709dc128-d4ea-4277-9abe-a052475ae78e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.345248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-709dc128-d4ea-4277-9abe-a052475ae78e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.345638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-709dc128-d4ea-4277-9abe-a052475ae78e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.379900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-709dc128-d4ea-4277-9abe-a052475ae78e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments/f784b82c-57ad-4c26-a15f-75cea76d79c3" status: 200 len: 194 microversion: 2.20 time: 0.048373 Jul 31 01:15:00.380549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 328/645] 10.222.0.42 () {60 vars in 1454 bytes} [Sun Jul 31 01:15:00 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments/f784b82c-57ad-4c26-a15f-75cea76d79c3 => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:15:00.456381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:00.457005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:00.463849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.464302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.465095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.465538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.607391 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lazy-loading 'services' on Instance uuid ef19c2df-372c-4c77-ad6c-ada1cd419a14 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:15:00.733242 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8f2df3f6-32c4-4d5d-8a79-c3e41e1775b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments/f784b82c-57ad-4c26-a15f-75cea76d79c3" status: 202 len: 0 microversion: 2.20 time: 0.281464 Jul 31 01:15:00.734127 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 318/646] 10.222.0.42 () {60 vars in 1457 bytes} [Sun Jul 31 01:15:00 2022] DELETE /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments/f784b82c-57ad-4c26-a15f-75cea76d79c3 => generated 0 bytes in 283 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:15:00.743715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f6a65f2e-517e-4c83-a3dd-c2cabc1d5431 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:00.746786 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f6a65f2e-517e-4c83-a3dd-c2cabc1d5431 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:00.755272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6a65f2e-517e-4c83-a3dd-c2cabc1d5431 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.755951 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6a65f2e-517e-4c83-a3dd-c2cabc1d5431 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.756959 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6a65f2e-517e-4c83-a3dd-c2cabc1d5431 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:00.757919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f6a65f2e-517e-4c83-a3dd-c2cabc1d5431 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:00.793709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f6a65f2e-517e-4c83-a3dd-c2cabc1d5431 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054714 Jul 31 01:15:00.794308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 329/647] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Jul 31 01:15:00 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:15:01.222998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:01.223822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:01.230745 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:01.231249 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:01.232140 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:01.232667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:01.390984 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:01.401834 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a43f3227-1db3-481d-a518-fe44c87d484e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.183403 Jul 31 01:15:01.402444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 319/648] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:01 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:01.810110 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-052b2dad-0473-43ee-a367-673637f9ae5d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:01.812360 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-052b2dad-0473-43ee-a367-673637f9ae5d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:01.819877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-052b2dad-0473-43ee-a367-673637f9ae5d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:01.820334 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-052b2dad-0473-43ee-a367-673637f9ae5d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:01.821135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-052b2dad-0473-43ee-a367-673637f9ae5d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:01.821620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-052b2dad-0473-43ee-a367-673637f9ae5d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:01.855943 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-052b2dad-0473-43ee-a367-673637f9ae5d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049931 Jul 31 01:15:01.856542 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 330/649] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Jul 31 01:15:01 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:15:02.427637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:02.428663 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:02.439888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:02.440665 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:02.442024 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:02.442790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:02.630468 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:02.643398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-10d97b34-020f-4df3-89c1-bdf0990c69b5 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.223209 Jul 31 01:15:02.644411 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 320/650] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:02 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:02.868603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0ca0a04f-e126-45f9-9228-b5402f1606ee tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:02.870887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0ca0a04f-e126-45f9-9228-b5402f1606ee tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:02.879532 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ca0a04f-e126-45f9-9228-b5402f1606ee tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:02.880048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ca0a04f-e126-45f9-9228-b5402f1606ee tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:02.881004 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ca0a04f-e126-45f9-9228-b5402f1606ee tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:02.881519 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ca0a04f-e126-45f9-9228-b5402f1606ee tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:02.916638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0ca0a04f-e126-45f9-9228-b5402f1606ee tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052214 Jul 31 01:15:02.917296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 331/651] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Jul 31 01:15:02 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:15:03.161615 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1ce55f42-56d2-4b5b-9848-1ac20a65c76f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:03.164021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1ce55f42-56d2-4b5b-9848-1ac20a65c76f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:03.170620 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ce55f42-56d2-4b5b-9848-1ac20a65c76f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.171130 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ce55f42-56d2-4b5b-9848-1ac20a65c76f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.172012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ce55f42-56d2-4b5b-9848-1ac20a65c76f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.173094 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1ce55f42-56d2-4b5b-9848-1ac20a65c76f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.304776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1ce55f42-56d2-4b5b-9848-1ac20a65c76f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/action" status: 202 len: 0 microversion: 2.1 time: 0.147152 Jul 31 01:15:03.306047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 321/652] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:15:03 2022] POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/action => generated 0 bytes in 150 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:15:03.314667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:03.315287 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:03.322554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.323006 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.323829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.324263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.524229 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:03.534596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9295b5b9-5a05-47a2-bfca-126f9d06612d tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1749 microversion: 2.1 time: 0.223782 Jul 31 01:15:03.535203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 332/653] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:03 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1749 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:03.667319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:03.668091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:03.674972 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.676652 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.678287 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.679393 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.839016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:03.842949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:03.850356 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f39b25ed-3784-4cc6-9949-e98b841897b2 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.187550 Jul 31 01:15:03.851019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 322/654] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:03 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:03.930615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5c8df471-0623-426e-a251-ab85d639ea73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:03.933012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5c8df471-0623-426e-a251-ab85d639ea73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:03.940763 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c8df471-0623-426e-a251-ab85d639ea73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.941211 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c8df471-0623-426e-a251-ab85d639ea73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.942010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c8df471-0623-426e-a251-ab85d639ea73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:03.942441 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c8df471-0623-426e-a251-ab85d639ea73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:03.995195 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5c8df471-0623-426e-a251-ab85d639ea73 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068618 Jul 31 01:15:03.996286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 333/655] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Jul 31 01:15:03 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:15:04.571401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:04.572118 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:04.581419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:04.581871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:04.582653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:04.583124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:04.776246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:04.785929 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-94f96792-ca74-467d-8e1a-28ae170d21be tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1749 microversion: 2.1 time: 0.218846 Jul 31 01:15:04.786457 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 323/656] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:04 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1749 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:04.879198 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:04.879868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:04.886610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:04.887066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:04.887902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:04.888348 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.015215 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1f27e017-9111-4025-bb3d-48236d5167f9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:05.019982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1f27e017-9111-4025-bb3d-48236d5167f9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:05.026961 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f27e017-9111-4025-bb3d-48236d5167f9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.027459 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f27e017-9111-4025-bb3d-48236d5167f9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.028334 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f27e017-9111-4025-bb3d-48236d5167f9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.028787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1f27e017-9111-4025-bb3d-48236d5167f9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.064364 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1f27e017-9111-4025-bb3d-48236d5167f9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.056877 Jul 31 01:15:05.064956 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 324/657] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Jul 31 01:15:05 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14/os-volume_attachments => generated 25 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:05.118734 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:05.128467 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d118360c-b9f0-4dc3-a668-851c0b09abfc tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1736 microversion: 2.1 time: 0.253921 Jul 31 01:15:05.129492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 334/658] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:04 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1736 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:05.153767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:05.154265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:05.161896 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.162345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.163124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.163599 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.191819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:15:05.286476 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-eab8381c-00f8-48e3-ada5-70eeed9a31ac tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 204 len: 0 microversion: 2.20 time: 0.136394 Jul 31 01:15:05.287371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 325/659] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Jul 31 01:15:05 2022] DELETE /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 31 01:15:05.300056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:05.301121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:05.312324 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.313615 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.314924 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.315704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.529610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:05.538652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:05.540352 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:05.541915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-62af3017-246f-4954-a55e-5c2182058b37 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.249182 Jul 31 01:15:05.542512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 335/660] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:05 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:05.810005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:05.810626 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:05.818302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.818858 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.819783 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:05.820382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:05.997036 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:06.006418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ea946b9a-3d5e-4a23-bd76-0a1b62a1b46f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1749 microversion: 2.1 time: 0.201079 Jul 31 01:15:06.007088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 326/661] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:05 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1749 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:06.153424 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:06.154160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:06.161695 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.162169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.162964 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.163433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.322223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:06.332264 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6765213b-f07a-4d64-a5d7-fe0e964db328 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1716 microversion: 2.1 time: 0.182745 Jul 31 01:15:06.333012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 336/662] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:06 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1716 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:06.457595 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:06.458101 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:06.464870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.465320 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.466098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.466534 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.566612 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:06.567247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:06.570780 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] HTTP exception thrown: Cannot 'detach_volume' instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 while it is in vm_state rescued Jul 31 01:15:06.573628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Request method failure captured: Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments/0815bc2c-8dad-4c45-93ce-5e1618fad030 HTTP/1.1 Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dZ2ZANicX0K5Ht7oF4q37bz3ry4asTie3_vAAkdfXMm75OrKX_x8Trt62dwbXNsF8s_DZQ_uJMxtIx3O8383U61kM2V-WQSwIut0aag7RRA-YQK94v4Eloyyy8GU4XvoIDt7mCs5Ik6R1dpfM2P_EvHaOxL4bU--FT8De4fXyZLnrw Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:15:06.573908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_3d6a76e846cb48bb91eab4da1c842f2b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 52e5a4deed99442e8eed62f334aa670d Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:15:06.575533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Cannot 'detach_volume' instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 while it is in vm_state rescued Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'server_id': '4a6e0502-05f5-40cb-98d9-4d0fc3d3c656', 'id': '0815bc2c-8dad-4c45-93ce-5e1618fad030'} Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:15:06.576401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 while it is in vm_state rescued Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 while it is in vm_state rescued Jul 31 01:15:06.577976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:06.579697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.579697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.579867 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Returning 409 to user: Cannot 'detach_volume' instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 while it is in vm_state rescued {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:15:06.579962 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.580062 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ef5c3f78-b8ea-4222-8488-b5f9ba560167 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "DELETE /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments/0815bc2c-8dad-4c45-93ce-5e1618fad030" status: 409 len: 152 microversion: 2.1 time: 0.123387 Jul 31 01:15:06.580062 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 327/663] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:15:06 2022] DELETE /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments/0815bc2c-8dad-4c45-93ce-5e1618fad030 => generated 152 bytes in 125 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 31 01:15:06.590394 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-96a45fab-cf68-4832-a52f-7ab2a5e5d63a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:06.591444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-96a45fab-cf68-4832-a52f-7ab2a5e5d63a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:06.602761 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96a45fab-cf68-4832-a52f-7ab2a5e5d63a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.603618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96a45fab-cf68-4832-a52f-7ab2a5e5d63a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.604986 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96a45fab-cf68-4832-a52f-7ab2a5e5d63a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.605703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96a45fab-cf68-4832-a52f-7ab2a5e5d63a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.713370 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-96a45fab-cf68-4832-a52f-7ab2a5e5d63a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "POST /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/action" status: 202 len: 0 microversion: 2.1 time: 0.129255 Jul 31 01:15:06.714537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 328/664] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:15:06 2022] POST /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/action => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:15:06.728697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:06.730162 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:06.734243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:06.741620 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.742373 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.742857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:06.743754 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:06.744431 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:06.744602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:06.745720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3546a4e9-d2a1-4782-b9d5-939ce6d00187 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.183166 Jul 31 01:15:06.746291 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 337/665] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:06 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:15:06.916506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:06.925652 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-446b5cd6-00bf-46de-864d-d06e52544c60 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1724 microversion: 2.1 time: 0.205305 Jul 31 01:15:06.926249 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 329/666] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:06 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1724 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:07.030336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:07.031201 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:07.039432 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:07.040203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:07.041546 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:07.042319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:07.322291 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:07.331318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2eb78caf-cc59-4b1b-9453-b25848443cfa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1749 microversion: 2.1 time: 0.306129 Jul 31 01:15:07.332040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 338/667] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:07 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1749 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:07.771070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:07.780455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:07.793837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:07.794730 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:07.795941 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:07.796709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:07.952171 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:07.953803 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:07.967818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:07.968622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:07.970933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:07.971772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:08.160626 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:08.175633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:08.177961 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:08.179760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e4229f8a-9fa2-44cd-b9af-8a4cdcb14fd4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1723 microversion: 2.20 time: 0.413933 Jul 31 01:15:08.180735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 330/668] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:07 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1723 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:08.355060 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:08.356019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:08.363822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:08.364548 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:08.365425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:08.365885 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:08.462983 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:08.479922 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-75679e1f-e2ad-46eb-8529-d07618b99e88 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1724 microversion: 2.1 time: 0.534422 Jul 31 01:15:08.480939 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 339/669] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:07 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1724 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:08.583403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:08.599867 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0e0084ba-ffd4-42f9-a7f7-c12e6a6949ee tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.249825 Jul 31 01:15:08.600970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 331/670] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:08 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:09.157040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-7d250247-1dd4-44a7-8e87-46abfb3c45d3 req-4a304326-6509-4cb6-91ed-3001ac6ee9b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ef19c2df-372c-4c77-ad6c-ada1cd419a14", "name": "network-vif-deleted", "tag": "f5779c70-0ae8-4a58-a8ba-25f9117da52e"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:09.169389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-7d250247-1dd4-44a7-8e87-46abfb3c45d3 req-4a304326-6509-4cb6-91ed-3001ac6ee9b1 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:09.169747 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-7d250247-1dd4-44a7-8e87-46abfb3c45d3 req-4a304326-6509-4cb6-91ed-3001ac6ee9b1 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:09.205649 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-7d250247-1dd4-44a7-8e87-46abfb3c45d3 req-4a304326-6509-4cb6-91ed-3001ac6ee9b1 service nova] Creating event network-vif-deleted:f5779c70-0ae8-4a58-a8ba-25f9117da52e for instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 on hv-cinder-maste Jul 31 01:15:09.207193 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:09.208453 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:09.213759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-7d250247-1dd4-44a7-8e87-46abfb3c45d3 req-4a304326-6509-4cb6-91ed-3001ac6ee9b1 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060213 Jul 31 01:15:09.216330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:09.217030 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:09.218776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:09.219632 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:09.219761 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 340/671] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:15:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:15:09.320781 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:09.338860 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:09.341933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:09.344293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-37f60fad-da36-4c15-aed5-5d3a78023757 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1656 microversion: 2.20 time: 0.142735 Jul 31 01:15:09.345343 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 332/672] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:09 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1656 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:15:09.508074 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:09.509393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:09.521140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:09.523044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:09.524766 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:09.525512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:09.628118 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:09.629170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:09.641910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:09.642770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:09.644302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:09.646586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:09.702556 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:09.718672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-adea0d75-716f-4789-ad72-3f97f9c750b8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1724 microversion: 2.1 time: 0.218484 Jul 31 01:15:09.720733 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 341/673] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:09 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1724 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:09.896275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:09.905837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3eb39095-de23-4556-b030-66dc9edd0b89 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.286448 Jul 31 01:15:09.906993 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 333/674] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:09 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:10.385782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:10.386732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:10.396169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:10.396833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:10.397828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:10.398356 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:10.419525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:10.477692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:10.487008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:10.488570 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:10.489758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3c310205-5364-4520-b305-f7595ecc0502 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1656 microversion: 2.20 time: 0.109234 Jul 31 01:15:10.490341 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 342/675] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:10 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1656 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:15:10.757854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:10.758861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:10.771989 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:10.772772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:10.774190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:10.774993 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:10.938494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:10.939746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:10.950985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:10.951860 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:10.953762 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:10.954490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:10.954868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:10.963974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-04c1988a-17e6-425a-987c-3b8be3842861 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1724 microversion: 2.1 time: 0.213779 Jul 31 01:15:10.964562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 334/676] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:10 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1724 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:11.411081 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:11.420778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7e5012ec-ff11-4679-a32c-ab22b503bd0b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.489943 Jul 31 01:15:11.421566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 343/677] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:10 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:11.514016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:11.514630 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:11.522165 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:11.522627 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:11.523529 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:11.523974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:11.560849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:11.561302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:11.561763 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lazy-loading 'fault' on Instance uuid ef19c2df-372c-4c77-ad6c-ada1cd419a14 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:15:11.607269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:11.616407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:11.617952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:11.619165 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-efa0f632-c06a-4b35-b019-ca8bf1e96dcb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 200 len: 1371 microversion: 2.20 time: 0.109239 Jul 31 01:15:11.619778 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 335/678] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:11 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 1371 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:11.987102 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:11.987791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:11.995100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:11.995854 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:11.996785 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:11.997285 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:12.150543 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:12.160067 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-49dd4e76-0af0-4fc7-9a30-468ae80ae582 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1730 microversion: 2.1 time: 0.177569 Jul 31 01:15:12.160745 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 344/679] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:11 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1730 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:12.182880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e700ae82-0f93-46f5-8044-6e5d87322a75 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:12.183522 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e700ae82-0f93-46f5-8044-6e5d87322a75 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:12.190426 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e700ae82-0f93-46f5-8044-6e5d87322a75 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:12.190926 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e700ae82-0f93-46f5-8044-6e5d87322a75 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:12.191894 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e700ae82-0f93-46f5-8044-6e5d87322a75 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:12.192391 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e700ae82-0f93-46f5-8044-6e5d87322a75 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:12.211969 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:12.281124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e700ae82-0f93-46f5-8044-6e5d87322a75 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.102212 Jul 31 01:15:12.282810 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 336/680] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:15:12 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-interface => generated 284 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:15:12.446596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:12.448311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:12.461448 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:12.462184 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:12.463612 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:12.464292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:12.645360 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:12.646709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:12.658067 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:12.659583 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:12.660726 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:12.661604 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:12.680007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] HTTP exception thrown: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Request method failure captured: Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 HTTP/1.1 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYE5CsTG8XAGRyb7_DMNnr9aW5_pey_uEysBzVl7GCzZbKB1Tms3eNompJRXqRjZEel38CmpCHlWf15YLg0A5CerD_bRG6LtuidRf7CvO4XxrxqeneoQvqNyWZoFZeZUbgGKZ07QkooErBFk3wzINIfyHq-pgQ54Be66TzjO0BDud4 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Openstack-Nova-Api-Version: 2.20 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:15:12.682121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/80ea98b340d94181b2193f4a79905988"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_80ea98b340d94181b2193f4a79905988", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 56732ebe8ee24f3f9ce8f87746e8b161 Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:15:12.683577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'ef19c2df-372c-4c77-ad6c-ada1cd419a14'} Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:15:12.684346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:15:12.685543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:15:12.686638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Returning 404 to user: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:15:12.688361 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-52e7dce5-a2e0-4654-8a63-1710bc4a817b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 404 len: 111 microversion: 2.20 time: 0.048776 Jul 31 01:15:12.689490 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 337/681] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:12 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 31 01:15:12.706515 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-272b4ae8-4fe5-42f1-9345-30192edcdcd0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '.version_select at 0x7f3e7b6ca550>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:12.723296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:12.736261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d52526df-d40c-45bd-ac4f-6a9b6906b41e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.296672 Jul 31 01:15:12.749159 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 345/682] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:12 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:12.749658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-272b4ae8-4fe5-42f1-9345-30192edcdcd0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-460569984" status: 204 len: 0 microversion: 2.20 time: 0.040977 Jul 31 01:15:12.749658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 338/683] 10.222.0.42 () {60 vars in 1259 bytes} [Sun Jul 31 01:15:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-460569984 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 31 01:15:13.467550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-069b99ae-a04c-4999-a596-a4e7ddde3bd5 req-4d037d50-1d8e-42a2-b02d-44fff777cf78 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4a6e0502-05f5-40cb-98d9-4d0fc3d3c656", "tag": "ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:13.481022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-069b99ae-a04c-4999-a596-a4e7ddde3bd5 req-4d037d50-1d8e-42a2-b02d-44fff777cf78 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:13.481797 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-069b99ae-a04c-4999-a596-a4e7ddde3bd5 req-4d037d50-1d8e-42a2-b02d-44fff777cf78 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:13.495635 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-069b99ae-a04c-4999-a596-a4e7ddde3bd5 req-4d037d50-1d8e-42a2-b02d-44fff777cf78 service nova] Creating event network-changed:ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc for instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 on hv-cinder-maste Jul 31 01:15:13.501818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-069b99ae-a04c-4999-a596-a4e7ddde3bd5 req-4d037d50-1d8e-42a2-b02d-44fff777cf78 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038072 Jul 31 01:15:13.503025 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 346/684] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:15:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:15:13.773127 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:13.774016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:13.780755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:13.781808 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:13.782882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:13.783539 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:13.908342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2ec9958c-9bcb-44de-89d7-4d16c53146c6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'create', calling method: .version_select at 0x7f3e7b7cd430>, body: {"keypair": {"name": "tempest-keypair-534147176", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBuUNyNPpkIVCAXl6+X518fMNufAewwOT3v7LiVawXBh4r6SN84aT6imXlvucJx90jYI1l4RWUzMIgx4YfRYhdlPP2i0LxA33+A34xDqWvBr5SeskAFpOTERaZHpiAHkFQ=="}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:13.928910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-2ec9958c-9bcb-44de-89d7-4d16c53146c6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for project 80ea98b340d94181b2193f4a79905988. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:15:13.934792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-2ec9958c-9bcb-44de-89d7-4d16c53146c6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for user 56732ebe8ee24f3f9ce8f87746e8b161 and project 80ea98b340d94181b2193f4a79905988. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:15:13.988507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2ec9958c-9bcb-44de-89d7-4d16c53146c6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.085346 Jul 31 01:15:13.991591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 347/685] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Jul 31 01:15:13 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 90 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Jul 31 01:15:14.079988 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:14.090319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6c1f1e44-f792-42d7-9356-636d6a1127ff tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.321250 Jul 31 01:15:14.091274 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 339/686] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:13 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:15.113858 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:15.114270 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:15.116402 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:15.123774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:15.124218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:15.125080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:15.125533 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:15.502655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:15.511911 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6aaa5ddf-823a-44a3-baef-3e76672189d8 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.403153 Jul 31 01:15:15.524628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 348/687] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:15 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:15.532194 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-7c29d927-8c1c-4f66-8616-549305f540ec req-35c81eeb-3500-4778-b857-0cd1a2b66c06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4a6e0502-05f5-40cb-98d9-4d0fc3d3c656", "tag": "ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:15.554224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:15.558844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-7c29d927-8c1c-4f66-8616-549305f540ec req-35c81eeb-3500-4778-b857-0cd1a2b66c06 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:15.559295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-7c29d927-8c1c-4f66-8616-549305f540ec req-35c81eeb-3500-4778-b857-0cd1a2b66c06 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:15.572456 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-7c29d927-8c1c-4f66-8616-549305f540ec req-35c81eeb-3500-4778-b857-0cd1a2b66c06 service nova] Creating event network-changed:ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc for instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 on hv-cinder-maste Jul 31 01:15:15.578537 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-7c29d927-8c1c-4f66-8616-549305f540ec req-35c81eeb-3500-4778-b857-0cd1a2b66c06 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056567 Jul 31 01:15:15.579325 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 340/688] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:15:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:15:16.467473 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-800507422", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "c3c043fa-9814-4058-ac46-ed026e54cf1b"}], "security_groups": [{"name": "tempest-securitygroup--716913124"}], "key_name": "tempest-keypair-534147176", "user_data": "}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:16.560415 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:16.561660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:16.574768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:16.575599 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:16.576957 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:16.577736 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:16.674185 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] validate_networks() for [('c3c043fa-9814-4058-ac46-ed026e54cf1b', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:15:16.799473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:16.808464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a13d6727-d75f-469e-9827-a3c74b0aad8e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.264751 Jul 31 01:15:16.809058 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 341/689] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:16 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:17.226421 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:15:17.226846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:15:17.227325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:15:17.227774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:15:17.228190 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:15:17.237592 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:17.238045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:17.238873 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:17.239313 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:17.260053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for project 80ea98b340d94181b2193f4a79905988. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:15:17.264822 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Getting quotas for user 56732ebe8ee24f3f9ce8f87746e8b161 and project 80ea98b340d94181b2193f4a79905988. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:15:17.279004 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:15:17.281877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:15:17.335795 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3fbaa768-38dd-479f-b7ed-79c88726673a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.876694 Jul 31 01:15:17.336449 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 349/690] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Jul 31 01:15:16 2022] POST /compute/v2.1/servers => generated 401 bytes in 879 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 31 01:15:17.352765 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ffa7e28e-f5e1-482f-9c5b-d857df108c85 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:17.353602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ffa7e28e-f5e1-482f-9c5b-d857df108c85 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:17.428497 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ffa7e28e-f5e1-482f-9c5b-d857df108c85 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:17.438294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ffa7e28e-f5e1-482f-9c5b-d857df108c85 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:17.440269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ffa7e28e-f5e1-482f-9c5b-d857df108c85 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:17.441625 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ffa7e28e-f5e1-482f-9c5b-d857df108c85 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1280 microversion: 2.20 time: 0.093637 Jul 31 01:15:17.442426 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 342/691] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:17 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1280 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:17.842464 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:17.844091 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:17.852755 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:17.853369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:17.854333 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:17.854949 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:18.036105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:18.045311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cf14c0a7-04c9-48ba-8f4e-8508cddbcaa4 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.208067 Jul 31 01:15:18.045904 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 350/692] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:17 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:18.469264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:18.469897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:18.478087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:18.478836 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:18.479844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:18.480405 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:18.586353 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:18.604674 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:18.606838 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:18.608267 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2d0b9a31-d02c-4dae-a546-556ec59a03c4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1340 microversion: 2.20 time: 0.143219 Jul 31 01:15:18.609960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 343/693] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:18 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1340 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:15:19.071523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:19.072955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:19.084204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:19.084687 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:19.085471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:19.085915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:19.450750 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:19.460948 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4a9b9778-3343-4519-967e-9c6380147ac9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.396847 Jul 31 01:15:19.461682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 351/694] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:19 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:19.638008 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:19.639387 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:19.652788 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:19.653977 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:19.656156 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:19.656974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:19.881610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:19.892004 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:19.893591 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:19.894798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-256791c6-2df3-4635-90a5-0fd34f4c33e6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1405 microversion: 2.20 time: 0.263761 Jul 31 01:15:19.895737 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 344/695] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:19 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1405 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:20.492721 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:20.493894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:20.504487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:20.505185 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:20.506359 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:20.507040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:20.817072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:20.826852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-36b0e371-1fe1-42fb-97b3-559e4e9276d3 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.339035 Jul 31 01:15:20.828528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 352/696] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:20 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:20.909474 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-90b2e684-ab24-46c7-8a1b-d016b276a827 req-8e1fa888-3b00-4394-89b7-d88566744484 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c03b1463-390f-430b-9bdf-64322c030504", "tag": "7d683e21-e88a-4b21-ba07-f00c5ff5031e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:20.928300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:20.930576 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-90b2e684-ab24-46c7-8a1b-d016b276a827 req-8e1fa888-3b00-4394-89b7-d88566744484 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:20.931556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-90b2e684-ab24-46c7-8a1b-d016b276a827 req-8e1fa888-3b00-4394-89b7-d88566744484 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:20.931758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:20.945419 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:20.946123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:20.947673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:20.948651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:20.953943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-90b2e684-ab24-46c7-8a1b-d016b276a827 req-8e1fa888-3b00-4394-89b7-d88566744484 service nova] Creating event network-changed:7d683e21-e88a-4b21-ba07-f00c5ff5031e for instance c03b1463-390f-430b-9bdf-64322c030504 on hv-cinder-maste Jul 31 01:15:20.961747 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-90b2e684-ab24-46c7-8a1b-d016b276a827 req-8e1fa888-3b00-4394-89b7-d88566744484 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057965 Jul 31 01:15:20.977749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 345/697] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:15:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:15:21.212370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:21.222960 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:21.226294 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:21.227924 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9e625979-c5bd-43d5-817e-de721f4fae66 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1405 microversion: 2.20 time: 0.306823 Jul 31 01:15:21.228616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 353/698] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:20 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1405 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:15:21.852531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:21.853244 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:21.861807 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:21.862335 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:21.863207 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:21.863831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:22.056066 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:22.067222 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-40277d5a-37b2-4990-92c6-c794bff1cb2c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.218974 Jul 31 01:15:22.067952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 346/699] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:21 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:15:22.266678 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:22.267310 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:22.274241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:22.274878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:22.275841 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:22.276373 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:22.433832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:22.444839 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:22.446449 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:22.447738 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-46d9f344-9045-42b8-8b8a-81b0790efc39 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1405 microversion: 2.20 time: 0.185563 Jul 31 01:15:22.448318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 354/700] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:22 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1405 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:23.091019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:23.091584 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:23.098638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:23.099106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:23.099909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:23.100350 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:23.254417 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:23.262864 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7db8a5fe-ff3d-4fe4-b5bb-c27ddfe501b6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.175725 Jul 31 01:15:23.263522 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 347/701] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:23 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:23.485927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:23.486686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:23.494153 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:23.494687 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:23.495607 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:23.496094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:23.644651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:23.653384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:23.654908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:23.656808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-46055ac5-101a-4f5e-8ef6-8454eaafe17c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1405 microversion: 2.20 time: 0.175847 Jul 31 01:15:23.657424 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 355/702] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:23 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1405 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:15:24.286006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:24.287157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:24.294638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:24.295132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:24.296000 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:24.296408 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:24.568700 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:24.582454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b9fd9240-96c7-4e4d-a4e6-5800eeb7af19 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1754 microversion: 2.1 time: 0.300884 Jul 31 01:15:24.583216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 348/703] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:24 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1754 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:24.682909 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:24.685183 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:24.694158 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:24.694789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:24.695816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:24.696377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:24.850921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:24.860984 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:24.862669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:24.863872 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-73a3b144-b112-46ff-b93a-c29b6963f5f6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1405 microversion: 2.20 time: 0.186182 Jul 31 01:15:24.864495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 356/704] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:24 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1405 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:25.606536 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:25.607165 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:25.614422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:25.614888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:25.615728 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:25.616171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:25.781052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:25.790577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c0398654-c52a-440d-b279-94844d1fccb6 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1757 microversion: 2.1 time: 0.188063 Jul 31 01:15:25.791268 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 349/705] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:15:25 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1757 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:15:25.886368 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:25.886990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:25.894602 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:25.895056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:25.895878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:25.896322 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:26.075034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:26.084301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:26.085831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:26.086986 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e51dafcf-b481-4d1d-ba6d-42db97d503fb tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1405 microversion: 2.20 time: 0.204618 Jul 31 01:15:26.087603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 357/706] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:25 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1405 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:15:26.519189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-83c5cbfd-4fe6-4449-8476-b2f504049bc6 req-ad395e54-97ba-4332-8299-364544307c76 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c03b1463-390f-430b-9bdf-64322c030504", "name": "network-vif-plugged", "status": "completed", "tag": "7d683e21-e88a-4b21-ba07-f00c5ff5031e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:26.532289 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-83c5cbfd-4fe6-4449-8476-b2f504049bc6 req-ad395e54-97ba-4332-8299-364544307c76 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:26.532828 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-83c5cbfd-4fe6-4449-8476-b2f504049bc6 req-ad395e54-97ba-4332-8299-364544307c76 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:26.546318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-83c5cbfd-4fe6-4449-8476-b2f504049bc6 req-ad395e54-97ba-4332-8299-364544307c76 service nova] Creating event network-vif-plugged:7d683e21-e88a-4b21-ba07-f00c5ff5031e for instance c03b1463-390f-430b-9bdf-64322c030504 on hv-cinder-maste Jul 31 01:15:26.552177 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-83c5cbfd-4fe6-4449-8476-b2f504049bc6 req-ad395e54-97ba-4332-8299-364544307c76 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037625 Jul 31 01:15:26.553279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 350/707] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:15:27.111017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:27.111802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:27.119129 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:27.119664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:27.120490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:27.120975 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:27.247567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:27.274274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=10 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:27.289770 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:27.306172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:27.309221 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:27.311423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2f101539-45e3-46bd-a44b-34ff80cc97da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1405 microversion: 2.20 time: 0.204692 Jul 31 01:15:27.312484 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 358/708] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:27 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1405 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:28.333410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:28.340795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:28.342224 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:28.352135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:28.352894 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:28.354749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:28.354749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:28.517803 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:28.527523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:28.529277 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:15:28.531427 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9d322984-1054-4a14-bdb3-4286d32954a3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1598 microversion: 2.20 time: 0.199227 Jul 31 01:15:28.532136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 351/709] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:15:28 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1598 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:15:28.554767 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f5175af6-08a9-4c94-a9fd-68c639eaf438 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:28.555308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f5175af6-08a9-4c94-a9fd-68c639eaf438 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:28.562366 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5175af6-08a9-4c94-a9fd-68c639eaf438 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:28.562799 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5175af6-08a9-4c94-a9fd-68c639eaf438 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:28.563627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5175af6-08a9-4c94-a9fd-68c639eaf438 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:28.564068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f5175af6-08a9-4c94-a9fd-68c639eaf438 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:28.648261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f5175af6-08a9-4c94-a9fd-68c639eaf438 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.097199 Jul 31 01:15:28.649187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 359/710] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Jul 31 01:15:28 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:15:29.804277 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-501b4c13-f6f1-4b05-8250-055b36bcfbb6 req-363eee77-40c6-460b-81cb-f7b0070661f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c03b1463-390f-430b-9bdf-64322c030504", "tag": "7d683e21-e88a-4b21-ba07-f00c5ff5031e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:29.817881 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-501b4c13-f6f1-4b05-8250-055b36bcfbb6 req-363eee77-40c6-460b-81cb-f7b0070661f6 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:29.818892 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-501b4c13-f6f1-4b05-8250-055b36bcfbb6 req-363eee77-40c6-460b-81cb-f7b0070661f6 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:29.833688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-501b4c13-f6f1-4b05-8250-055b36bcfbb6 req-363eee77-40c6-460b-81cb-f7b0070661f6 service nova] Creating event network-changed:7d683e21-e88a-4b21-ba07-f00c5ff5031e for instance c03b1463-390f-430b-9bdf-64322c030504 on hv-cinder-maste Jul 31 01:15:29.839717 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-501b4c13-f6f1-4b05-8250-055b36bcfbb6 req-363eee77-40c6-460b-81cb-f7b0070661f6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039739 Jul 31 01:15:29.840737 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 352/711] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:15:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:15:54.642645 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:54.643385 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=89 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:54.647584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1976b201-2524-4c5d-9db7-817589016b82 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:15:54.648091 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1976b201-2524-4c5d-9db7-817589016b82 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:54.656095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1976b201-2524-4c5d-9db7-817589016b82 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:54.656555 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1976b201-2524-4c5d-9db7-817589016b82 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:54.657368 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1976b201-2524-4c5d-9db7-817589016b82 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:54.657810 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1976b201-2524-4c5d-9db7-817589016b82 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:54.686199 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1976b201-2524-4c5d-9db7-817589016b82 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/ips" status: 200 len: 150 microversion: 2.1 time: 0.044577 Jul 31 01:15:54.686817 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 360/712] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:15:54 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/ips => generated 150 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:15:56.128556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-16baf101-8b5a-4525-bd2c-9f2534962588 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a8d8fc20-00a4-4c58-9ad2-430890854388"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:15:56.131174 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-16baf101-8b5a-4525-bd2c-9f2534962588 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:15:56.133625 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=27 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:56.134287 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=64 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:15:56.141836 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16baf101-8b5a-4525-bd2c-9f2534962588 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:56.142290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16baf101-8b5a-4525-bd2c-9f2534962588 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:56.143072 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16baf101-8b5a-4525-bd2c-9f2534962588 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:15:56.143802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-16baf101-8b5a-4525-bd2c-9f2534962588 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:15:56.496499 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-16baf101-8b5a-4525-bd2c-9f2534962588 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.372360 Jul 31 01:15:56.497149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 353/713] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:15:56 2022] POST /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments => generated 194 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:04.197186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a8d8fc20-00a4-4c58-9ad2-430890854388"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:04.199857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:04.207813 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:04.208266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:04.209061 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:04.209503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:04.304447 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Invalid volume: volume a8d8fc20-00a4-4c58-9ad2-430890854388 is already attached to instances: d1cf1b41-8f07-4466-b084-79a321360da3 Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: POST /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments HTTP/1.1 Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 74 Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:16:04.306298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {"volumeAttachment": {"volumeId": "a8d8fc20-00a4-4c58-9ad2-430890854388"}} Jul 31 01:16:04.308829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Invalid volume: volume a8d8fc20-00a4-4c58-9ad2-430890854388 is already attached to instances: d1cf1b41-8f07-4466-b084-79a321360da3 Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'server_id': 'd1cf1b41-8f07-4466-b084-79a321360da3', 'body': {'volumeAttachment': {'volumeId': 'a8d8fc20-00a4-4c58-9ad2-430890854388'}}} Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5066, in attach_volume Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4906, in _check_volume_already_attached Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume a8d8fc20-00a4-4c58-9ad2-430890854388 is already attached to instances: d1cf1b41-8f07-4466-b084-79a321360da3 Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:04.310196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume a8d8fc20-00a4-4c58-9ad2-430890854388 is already attached to instances: d1cf1b41-8f07-4466-b084-79a321360da3 Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:04.312378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume a8d8fc20-00a4-4c58-9ad2-430890854388 is already attached to instances: d1cf1b41-8f07-4466-b084-79a321360da3 Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 400 to user: Invalid volume: volume a8d8fc20-00a4-4c58-9ad2-430890854388 is already attached to instances: d1cf1b41-8f07-4466-b084-79a321360da3 {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-beea3fe8-6a4c-4e58-b043-314b20b634d9 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.114746 Jul 31 01:16:04.314675 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 361/714] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:16:04 2022] POST /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments => generated 174 bytes in 116 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 31 01:16:04.402880 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:04.403613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:04.411132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:04.411628 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:04.412459 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:04.412934 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:04.501856 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lazy-loading 'services' on Instance uuid d1cf1b41-8f07-4466-b084-79a321360da3 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:04.594092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1537c6df-cba0-4837-91ba-2b41e31d215f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments/a8d8fc20-00a4-4c58-9ad2-430890854388" status: 202 len: 0 microversion: 2.1 time: 0.197609 Jul 31 01:16:04.595189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 354/715] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:16:04 2022] DELETE /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments/a8d8fc20-00a4-4c58-9ad2-430890854388 => generated 0 bytes in 201 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:04.608766 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-021d2d64-ffb0-4876-b7ba-2d75718fa43f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:04.613169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-021d2d64-ffb0-4876-b7ba-2d75718fa43f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:04.625042 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-021d2d64-ffb0-4876-b7ba-2d75718fa43f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:04.625791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-021d2d64-ffb0-4876-b7ba-2d75718fa43f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:04.626999 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-021d2d64-ffb0-4876-b7ba-2d75718fa43f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:04.627683 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-021d2d64-ffb0-4876-b7ba-2d75718fa43f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:04.666912 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-021d2d64-ffb0-4876-b7ba-2d75718fa43f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066474 Jul 31 01:16:04.667550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 362/716] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:04 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:05.682660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cd3994e8-060d-4ada-a236-0de57ed4f017 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:05.685189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cd3994e8-060d-4ada-a236-0de57ed4f017 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:05.692342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd3994e8-060d-4ada-a236-0de57ed4f017 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:05.692786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd3994e8-060d-4ada-a236-0de57ed4f017 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:05.693600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd3994e8-060d-4ada-a236-0de57ed4f017 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:05.694235 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cd3994e8-060d-4ada-a236-0de57ed4f017 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:05.729007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cd3994e8-060d-4ada-a236-0de57ed4f017 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051495 Jul 31 01:16:05.729607 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 355/717] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:05 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 31 01:16:05.829388 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:05.830219 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:05.837406 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:05.837773 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:05.838572 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:05.839014 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:05.868513 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:16:05.966562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b47aaac6-a889-4b0a-bb47-c1bf0532466f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 204 len: 0 microversion: 2.1 time: 0.141952 Jul 31 01:16:05.967166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 363/718] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:16:05 2022] DELETE /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:16:05.975994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:05.976634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:05.983218 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:05.983718 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:05.984514 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:05.984956 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:06.140971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:06.149375 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6ad2f835-564a-4e06-876d-6aa25cb30d1d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1680 microversion: 2.1 time: 0.177229 Jul 31 01:16:06.149986 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 356/719] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:05 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1680 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:07.173200 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:07.174109 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:07.181754 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:07.182206 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:07.183104 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:07.183748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:07.345301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:07.353880 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-151781b7-adfa-4a5d-a772-20e0ef665158 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1680 microversion: 2.1 time: 0.185701 Jul 31 01:16:07.354625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 364/720] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:07 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1680 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:08.377841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:08.379274 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:08.389422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:08.390442 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:08.391633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:08.392253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:08.615079 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:08.627018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b50705c2-4c44-4d29-894c-4c7ab80d8874 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1680 microversion: 2.1 time: 0.253010 Jul 31 01:16:08.628721 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 357/721] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:08 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1680 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:09.087487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-028b8d62-c18f-4f48-a840-6ba97a1003df req-a52aa1c5-86a1-496b-a5d8-692a183a49db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1cf1b41-8f07-4466-b084-79a321360da3", "name": "network-vif-deleted", "tag": "59fd75bb-d9d0-4f3d-883e-d97aed7927ed"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:09.104459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-028b8d62-c18f-4f48-a840-6ba97a1003df req-a52aa1c5-86a1-496b-a5d8-692a183a49db service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:09.105211 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-028b8d62-c18f-4f48-a840-6ba97a1003df req-a52aa1c5-86a1-496b-a5d8-692a183a49db service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:09.131044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-028b8d62-c18f-4f48-a840-6ba97a1003df req-a52aa1c5-86a1-496b-a5d8-692a183a49db service nova] Creating event network-vif-deleted:59fd75bb-d9d0-4f3d-883e-d97aed7927ed for instance d1cf1b41-8f07-4466-b084-79a321360da3 on hv-cinder-maste Jul 31 01:16:09.142476 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-028b8d62-c18f-4f48-a840-6ba97a1003df req-a52aa1c5-86a1-496b-a5d8-692a183a49db service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063205 Jul 31 01:16:09.143915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 365/722] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:09.654498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:09.655412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:09.662515 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:09.662978 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:09.663822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:09.664269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:09.702595 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:09.703111 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:09.703624 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lazy-loading 'fault' on Instance uuid d1cf1b41-8f07-4466-b084-79a321360da3 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:09.771869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:09.787677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d6eba139-ede3-4865-a8f8-654aff5ae4bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 200 len: 1330 microversion: 2.1 time: 0.136985 Jul 31 01:16:09.788503 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 358/723] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:09 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 1330 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:10.830033 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:10.830646 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:10.838138 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:10.838636 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:10.839514 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:10.839991 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:10.855457 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 HTTP/1.1 Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:16:10.857706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:16:10.860125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'd1cf1b41-8f07-4466-b084-79a321360da3'} Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:10.861591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:16:10.863940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:10.865933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:10.865933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2b97796b-dc2e-4be3-9e99-6c619edf4f85 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 404 len: 111 microversion: 2.1 time: 0.033779 Jul 31 01:16:10.865933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 366/724] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:10 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:16:10.873241 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b9e039a9-bacf-4c07-95a9-de7dea09ebaf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '.version_select at 0x7f3e7b619310>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:10.903543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b9e039a9-bacf-4c07-95a9-de7dea09ebaf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-879799212" status: 202 len: 0 microversion: 2.1 time: 0.038201 Jul 31 01:16:10.904152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 359/725] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Jul 31 01:16:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-879799212 => generated 0 bytes in 40 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:11.716198 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5e09704b-e2bc-46e0-b4a9-0029194cab0e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: .version_select at 0x7f3e7b6e6670>, body: {"keypair": {"name": "tempest-keypair-1297203599", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBRiFQfmQ0/KunYmndKk9Bo0XBrdp0YZ8i1oOPQ1uQYRRKrxE2pHpa8c1LJoxqwNHcfh5+bjPCNugbUKp6A3pJZSl4Xts5Hd2CU4R4t8rsEqh2llzsiE9SXW03QhU8rbwA=="}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:11.734384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-5e09704b-e2bc-46e0-b4a9-0029194cab0e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:16:11.739275 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-5e09704b-e2bc-46e0-b4a9-0029194cab0e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:16:11.765853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5e09704b-e2bc-46e0-b4a9-0029194cab0e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.053798 Jul 31 01:16:11.766627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 367/726] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:16:11 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:13.236289 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1615607594", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b"}], "security_groups": [{"name": "tempest-securitygroup--871336880"}], "key_name": "tempest-keypair-1297203599", "user_data": "}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:13.393666 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] validate_networks() for [('3a9b4c68-f4f7-44c4-90ab-ef72e5a2c66b', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:16:14.108696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:16:14.109005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:16:14.109482 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:16:14.109918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:16:14.110329 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:16:14.120287 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:14.120673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:14.121488 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:14.122029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:14.142992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:16:14.148414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Getting quotas for user 20df8d74c87143dd81dbaae76b3c2b62 and project b94a935febe14fc5b0bda4ca88f78ca8. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:16:14.161092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:16:14.163635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:16:14.221434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-23d4baa9-1f30-4305-888b-24fc46aed563 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.993008 Jul 31 01:16:14.222219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 360/727] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:16:13 2022] POST /compute/v2.1/servers => generated 401 bytes in 995 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:16:14.236654 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9834beeb-8f8d-4766-9bb5-35222bd79eec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:14.237561 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9834beeb-8f8d-4766-9bb5-35222bd79eec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:14.322608 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9834beeb-8f8d-4766-9bb5-35222bd79eec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:14.330706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9834beeb-8f8d-4766-9bb5-35222bd79eec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1242 microversion: 2.1 time: 0.099901 Jul 31 01:16:14.331566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 368/728] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:14 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1242 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:15.353411 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:15.354054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:15.363574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:15.364029 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:15.364833 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:15.365847 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:15.442959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:15.451115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a7934016-2ef3-4dbc-8ae6-cdb5487ba011 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1302 microversion: 2.1 time: 0.102308 Jul 31 01:16:15.451719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 361/729] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:15 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1302 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:16.483497 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:16.484730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:16.491842 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:16.492667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:16.493652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:16.494086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:16.794345 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:16.806422 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-31c41873-d672-4b74-a2ed-8eff5f90a290 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.327085 Jul 31 01:16:16.807860 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 369/730] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:16 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:17.119411 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:17.120380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:17.132672 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:17.133500 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:17.134974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:17.135819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:17.244358 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lazy-loading 'services' on Instance uuid 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:17.258900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-f726cc5a-b018-4549-8bc1-fda95a6a70e5 req-24ace224-0f07-4fbd-8f38-89ff896af089 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29c5be43-d161-49e2-8214-6fff60151b8b", "tag": "658533d1-a639-468a-a657-a8f9bc1703b3"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:17.270096 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-f726cc5a-b018-4549-8bc1-fda95a6a70e5 req-24ace224-0f07-4fbd-8f38-89ff896af089 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:17.270638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-f726cc5a-b018-4549-8bc1-fda95a6a70e5 req-24ace224-0f07-4fbd-8f38-89ff896af089 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:17.284953 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-f726cc5a-b018-4549-8bc1-fda95a6a70e5 req-24ace224-0f07-4fbd-8f38-89ff896af089 service nova] Creating event network-changed:658533d1-a639-468a-a657-a8f9bc1703b3 for instance 29c5be43-d161-49e2-8214-6fff60151b8b on hv-cinder-maste Jul 31 01:16:17.291249 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-f726cc5a-b018-4549-8bc1-fda95a6a70e5 req-24ace224-0f07-4fbd-8f38-89ff896af089 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036894 Jul 31 01:16:17.291993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 370/731] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:17.360579 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-45be25fe-1cb2-4cbf-aded-619955737d48 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "DELETE /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments/0815bc2c-8dad-4c45-93ce-5e1618fad030" status: 202 len: 0 microversion: 2.1 time: 0.248786 Jul 31 01:16:17.361743 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 362/732] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:16:17 2022] DELETE /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments/0815bc2c-8dad-4c45-93ce-5e1618fad030 => generated 0 bytes in 251 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:17.371332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-183ab5e9-d63c-46e6-b872-2f4cb0d9641e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:17.373757 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-183ab5e9-d63c-46e6-b872-2f4cb0d9641e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:17.383756 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-183ab5e9-d63c-46e6-b872-2f4cb0d9641e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:17.384619 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-183ab5e9-d63c-46e6-b872-2f4cb0d9641e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:17.385523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-183ab5e9-d63c-46e6-b872-2f4cb0d9641e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:17.385963 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-183ab5e9-d63c-46e6-b872-2f4cb0d9641e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:17.422567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-183ab5e9-d63c-46e6-b872-2f4cb0d9641e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055745 Jul 31 01:16:17.423161 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 371/733] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:17 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:17.841316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:17.841908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:17.848955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:17.849414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:17.850209 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:17.850653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:18.022462 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:18.037031 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0822df18-e10f-4b5c-8ecc-5e6b222ae102 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.199766 Jul 31 01:16:18.038097 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 363/734] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:17 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:18.440649 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-02c9393f-62c9-41ad-a951-a3abf97d771e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:18.442877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-02c9393f-62c9-41ad-a951-a3abf97d771e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:18.450942 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02c9393f-62c9-41ad-a951-a3abf97d771e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:18.451508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02c9393f-62c9-41ad-a951-a3abf97d771e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:18.452364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02c9393f-62c9-41ad-a951-a3abf97d771e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:18.452865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02c9393f-62c9-41ad-a951-a3abf97d771e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:18.487336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-02c9393f-62c9-41ad-a951-a3abf97d771e tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050852 Jul 31 01:16:18.487963 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 372/735] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:18 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:19.064855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:19.065516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:19.073552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:19.074055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:19.075632 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:19.076183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:19.233404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:19.242113 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-025e1015-a7b0-4d2a-a284-a3bc20ed66a1 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.181066 Jul 31 01:16:19.242746 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 364/736] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:19 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:19.500332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-982baca1-4053-4b16-b3ec-670b39cda186 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:19.502550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-982baca1-4053-4b16-b3ec-670b39cda186 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:19.509737 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-982baca1-4053-4b16-b3ec-670b39cda186 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:19.510192 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-982baca1-4053-4b16-b3ec-670b39cda186 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:19.510995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-982baca1-4053-4b16-b3ec-670b39cda186 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:19.511510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-982baca1-4053-4b16-b3ec-670b39cda186 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:19.548397 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-982baca1-4053-4b16-b3ec-670b39cda186 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052089 Jul 31 01:16:19.548997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 373/737] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:19 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:20.264448 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:20.265266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:20.272767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:20.273279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:20.274126 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:20.274616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:20.433264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:20.441317 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-44c60b0b-9bd4-42f2-96cf-df05853828ce tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.181535 Jul 31 01:16:20.441928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 365/738] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:20 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:20.562071 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a8bd4018-9262-465a-a1ca-474a05adfe98 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:20.564312 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a8bd4018-9262-465a-a1ca-474a05adfe98 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:20.571962 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a8bd4018-9262-465a-a1ca-474a05adfe98 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:20.572413 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a8bd4018-9262-465a-a1ca-474a05adfe98 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:20.573202 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a8bd4018-9262-465a-a1ca-474a05adfe98 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:20.573643 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a8bd4018-9262-465a-a1ca-474a05adfe98 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:20.619275 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a8bd4018-9262-465a-a1ca-474a05adfe98 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060874 Jul 31 01:16:20.620008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 374/739] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:20 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:21.462144 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:21.462788 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:21.470753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.471227 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:21.472039 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.472475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:21.635981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:21.636454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1ff97a86-06f4-4550-b30e-59964454c65a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:21.640663 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1ff97a86-06f4-4550-b30e-59964454c65a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:21.644768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2c9bfcce-165c-4062-96aa-7576c716ba0a tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.186476 Jul 31 01:16:21.645653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 366/740] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:21 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:21.651822 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ff97a86-06f4-4550-b30e-59964454c65a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.652725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ff97a86-06f4-4550-b30e-59964454c65a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:21.654608 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ff97a86-06f4-4550-b30e-59964454c65a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.655427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ff97a86-06f4-4550-b30e-59964454c65a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:21.713295 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1ff97a86-06f4-4550-b30e-59964454c65a tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.083837 Jul 31 01:16:21.714298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 375/741] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:21 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656/os-volume_attachments => generated 25 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 31 01:16:21.827080 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:21.828065 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:21.838957 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.839805 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:21.841704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.842437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:21.874916 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:16:21.892057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:21.976218 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-799a61d1-8d14-4d43-b37f-b5aa6c4a876d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "DELETE /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 204 len: 0 microversion: 2.1 time: 0.156018 Jul 31 01:16:21.977021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 367/742] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:16:21 2022] DELETE /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:16:21.986338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:21.987023 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:21.994369 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.994916 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:21.995842 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:21.996368 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:22.149820 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:22.158047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-06bd0e65-b1e5-46e5-9b58-89b534eea906 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1690 microversion: 2.1 time: 0.176022 Jul 31 01:16:22.158874 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 376/743] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:21 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1690 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:22.671085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:22.672488 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:22.684855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:22.685640 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:22.687063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:22.687982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:22.871760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:22.882548 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fa2495af-ac8c-4083-9034-a2702393025f tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.218984 Jul 31 01:16:22.887554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 368/744] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:22 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:23.181935 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:23.182730 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:23.191247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:23.191791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:23.192658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:23.193849 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:23.354670 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:23.363233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0b9c916a-b361-46e3-b9b3-51b5cb4d8206 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1690 microversion: 2.1 time: 0.185874 Jul 31 01:16:23.363876 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 377/745] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:23 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1690 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:23.905212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:23.905827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:23.913096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:23.913591 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:23.914454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:23.915133 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:24.086044 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:24.094521 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-23b70101-22e1-4691-8678-326f475b1943 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.193179 Jul 31 01:16:24.095115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 369/746] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:23 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:24.384731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:24.385617 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:24.388512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:24.398314 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:24.399122 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:24.400688 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:24.401463 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:24.608415 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:24.623092 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4e21c45e-49e0-473d-a057-e1bbcd265d46 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1690 microversion: 2.1 time: 0.241948 Jul 31 01:16:24.623982 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 378/747] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:24 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1690 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:24.813777 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-6a3f44a0-f3fb-404e-b200-9068694bf09d req-cc0061bf-3f75-4415-b8a6-626f33cfe4ca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29c5be43-d161-49e2-8214-6fff60151b8b", "name": "network-vif-plugged", "status": "completed", "tag": "658533d1-a639-468a-a657-a8f9bc1703b3"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:24.822943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:24.823996 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=6 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:24.835584 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-6a3f44a0-f3fb-404e-b200-9068694bf09d req-cc0061bf-3f75-4415-b8a6-626f33cfe4ca service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:24.836226 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-6a3f44a0-f3fb-404e-b200-9068694bf09d req-cc0061bf-3f75-4415-b8a6-626f33cfe4ca service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:24.860551 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-6a3f44a0-f3fb-404e-b200-9068694bf09d req-cc0061bf-3f75-4415-b8a6-626f33cfe4ca service nova] Creating event network-vif-plugged:658533d1-a639-468a-a657-a8f9bc1703b3 for instance 29c5be43-d161-49e2-8214-6fff60151b8b on hv-cinder-maste Jul 31 01:16:24.868973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-6a3f44a0-f3fb-404e-b200-9068694bf09d req-cc0061bf-3f75-4415-b8a6-626f33cfe4ca service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062792 Jul 31 01:16:24.869608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 370/748] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:25.129255 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:25.130212 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:25.132812 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=55,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:25.140070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:25.140712 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:25.141672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:25.142234 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:25.148929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:25.347900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:25.356029 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b6132a3b-f7fd-4936-940c-253b516278e2 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.232373 Jul 31 01:16:25.356630 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 379/749] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:25 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:25.658106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:25.659057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:25.666925 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:25.667429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:25.668246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:25.668686 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:25.844525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:25.853656 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d5f124d3-e036-4426-bb44-92021458e1bb tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1690 microversion: 2.1 time: 0.200230 Jul 31 01:16:25.854270 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 371/750] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:25 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1690 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:26.377642 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:26.378358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:26.385178 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:26.385621 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:26.386397 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:26.386828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:26.536376 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:26.544969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e123dfd3-df6e-4c07-8dde-9b48d37691cf tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1367 microversion: 2.1 time: 0.171233 Jul 31 01:16:26.545566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 380/751] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:26 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1367 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:26.882645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:26.883645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:26.887863 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=60,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:26.896393 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:26.897395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:26.898505 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:26.899315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:27.100823 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:27.109967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-33cad08e-f687-4f05-8325-4cf227ba056d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1690 microversion: 2.1 time: 0.235190 Jul 31 01:16:27.110675 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 372/752] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:26 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1690 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:27.582760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:27.589002 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:27.606609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:27.608306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:27.610204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:27.611216 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:27.989272 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:28.001002 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3edac4e8-7584-4c90-b973-0220ee2bf3bd tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1557 microversion: 2.1 time: 0.426205 Jul 31 01:16:28.001985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 381/753] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:27 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1557 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:28.023471 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8e3858a1-8eac-4d05-aeb1-c5c0ef2cfd30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:28.024021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8e3858a1-8eac-4d05-aeb1-c5c0ef2cfd30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:28.031028 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e3858a1-8eac-4d05-aeb1-c5c0ef2cfd30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:28.031528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e3858a1-8eac-4d05-aeb1-c5c0ef2cfd30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:28.032345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e3858a1-8eac-4d05-aeb1-c5c0ef2cfd30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:28.032796 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e3858a1-8eac-4d05-aeb1-c5c0ef2cfd30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:28.139394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-1a02e15b-c9f0-4972-9917-b7f5624c4733 req-256eed2c-2174-4265-b22c-a27ce067a28b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4a6e0502-05f5-40cb-98d9-4d0fc3d3c656", "name": "network-vif-deleted", "tag": "ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:28.151061 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-1a02e15b-c9f0-4972-9917-b7f5624c4733 req-256eed2c-2174-4265-b22c-a27ce067a28b service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:28.151551 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-1a02e15b-c9f0-4972-9917-b7f5624c4733 req-256eed2c-2174-4265-b22c-a27ce067a28b service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:28.166286 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8e3858a1-8eac-4d05-aeb1-c5c0ef2cfd30 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.147241 Jul 31 01:16:28.167784 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 373/754] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:16:28 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-interface => generated 284 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:28.178407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:28.180060 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:28.187596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-1a02e15b-c9f0-4972-9917-b7f5624c4733 req-256eed2c-2174-4265-b22c-a27ce067a28b service nova] Creating event network-vif-deleted:ae64ca4d-eb66-4a81-9bf6-58f8a8fa8adc for instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 on hv-cinder-maste Jul 31 01:16:28.194603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-1a02e15b-c9f0-4972-9917-b7f5624c4733 req-256eed2c-2174-4265-b22c-a27ce067a28b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061980 Jul 31 01:16:28.196471 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:28.198304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:28.199166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 382/755] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:28.201174 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:28.202291 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:28.326381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:28.341919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-964fa1da-1937-47c8-aa5e-ea12dd08b202 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 200 len: 1331 microversion: 2.1 time: 0.172020 Jul 31 01:16:28.343793 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 374/756] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:28 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 1331 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:29.379433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:29.384704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:29.396174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:29.396978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:29.398913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:29.399715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:29.430160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] HTTP exception thrown: Instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 could not be found. Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Request method failure captured: Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 HTTP/1.1 Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dZ2ZANicX0K5Ht7oF4q37bz3ry4asTie3_vAAkdfXMm75OrKX_x8Trt62dwbXNsF8s_DZQ_uJMxtIx3O8383U61kM2V-WQSwIut0aag7RRA-YQK94v4Eloyyy8GU4XvoIDt7mCs5Ik6R1dpfM2P_EvHaOxL4bU--FT8De4fXyZLnrw Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:16:29.434060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_3d6a76e846cb48bb91eab4da1c842f2b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 52e5a4deed99442e8eed62f334aa670d Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 could not be found. Jul 31 01:16:29.435855 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '4a6e0502-05f5-40cb-98d9-4d0fc3d3c656'} Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 could not be found. Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:29.436798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 could not be found. Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 could not be found. Jul 31 01:16:29.438327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:29.439707 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Returning 404 to user: Instance 4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:29.439707 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2b6c4d42-4e31-40ac-a4ed-5432731a6139 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656" status: 404 len: 111 microversion: 2.1 time: 0.073214 Jul 31 01:16:29.440174 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 383/757] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:29 2022] GET /compute/v2.1/servers/4a6e0502-05f5-40cb-98d9-4d0fc3d3c656 => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:16:29.447958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-22005800-5259-4229-b3ed-960cff1f619f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '.version_select at 0x7f3e800cd3a0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:29.471812 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-22005800-5259-4229-b3ed-960cff1f619f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1282063812" status: 202 len: 0 microversion: 2.1 time: 0.027592 Jul 31 01:16:29.472899 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 375/758] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Jul 31 01:16:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1282063812 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:30.278816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-78ec5bb1-e9c6-4e27-a388-92e1862205d9 req-e52ee75e-1137-46c8-8245-1126d3ff7b38 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29c5be43-d161-49e2-8214-6fff60151b8b", "tag": "658533d1-a639-468a-a657-a8f9bc1703b3"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:30.310048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-78ec5bb1-e9c6-4e27-a388-92e1862205d9 req-e52ee75e-1137-46c8-8245-1126d3ff7b38 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:30.311076 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-78ec5bb1-e9c6-4e27-a388-92e1862205d9 req-e52ee75e-1137-46c8-8245-1126d3ff7b38 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:30.363859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-78ec5bb1-e9c6-4e27-a388-92e1862205d9 req-e52ee75e-1137-46c8-8245-1126d3ff7b38 service nova] Creating event network-changed:658533d1-a639-468a-a657-a8f9bc1703b3 for instance 29c5be43-d161-49e2-8214-6fff60151b8b on hv-cinder-maste Jul 31 01:16:30.376529 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-78ec5bb1-e9c6-4e27-a388-92e1862205d9 req-e52ee75e-1137-46c8-8245-1126d3ff7b38 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.105767 Jul 31 01:16:30.377182 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 384/759] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:32.752945 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:32.753803 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:32.760884 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:32.761335 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:32.762698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:32.763154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:32.793500 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:16:32.892475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-eba3fe79-d0bb-45a5-b66f-83a2f4c6d95f tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "DELETE /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 204 len: 0 microversion: 2.1 time: 0.143246 Jul 31 01:16:32.893389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 376/760] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:16:32 2022] DELETE /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:16:32.906300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:32.906995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:32.914418 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:32.914957 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:32.916159 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:32.916763 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.129040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:33.138413 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2c698942-96c4-4af1-896e-f4b898763769 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1509 microversion: 2.1 time: 0.237803 Jul 31 01:16:33.139055 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 385/761] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:32 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1509 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:33.207092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-386677154"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:33.207736 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:33.214930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:33.215429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.216214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:33.216648 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.381732 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.network.security_group_api [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Removing security group 6f0c7be0-f500-456b-b9a9-a904ad09a297 from port f704de2f-7bdb-4ef2-80b7-29cb6b9472eb Jul 31 01:16:33.477285 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:33.478119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:33.486441 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:33.487032 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.487859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:33.488295 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.514775 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lazy-loading 'flavor' on Instance uuid 69cf3658-0cb6-48b8-b152-72b915385fbb {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:33.658268 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e72add95-2c24-4d0a-8a07-5a60f68ee4c9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "POST /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb/action" status: 202 len: 0 microversion: 2.1 time: 0.186068 Jul 31 01:16:33.658992 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 386/762] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:16:33 2022] POST /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb/action => generated 0 bytes in 188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 31 01:16:33.668604 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:33.669298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:33.676563 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:33.677003 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.678944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:33.678944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.840875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-532a3648-6753-49c5-8b3b-12225a6fac9b tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/action" status: 202 len: 0 microversion: 2.1 time: 0.636977 Jul 31 01:16:33.844181 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 377/763] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:16:33 2022] POST /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/action => generated 0 bytes in 643 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:33.850149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-e0222d80-88b3-4eb1-9e12-37627b033f5e req-36163e22-778e-4547-b794-a51823da46e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7a3b72f2-b29c-4115-b742-1fa4a86e06b6", "tag": "f704de2f-7bdb-4ef2-80b7-29cb6b9472eb"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:33.870347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-e0222d80-88b3-4eb1-9e12-37627b033f5e req-36163e22-778e-4547-b794-a51823da46e1 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:33.870802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-e0222d80-88b3-4eb1-9e12-37627b033f5e req-36163e22-778e-4547-b794-a51823da46e1 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:33.888113 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-e0222d80-88b3-4eb1-9e12-37627b033f5e req-36163e22-778e-4547-b794-a51823da46e1 service nova] Creating event network-changed:f704de2f-7bdb-4ef2-80b7-29cb6b9472eb for instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 on hv-cinder-maste Jul 31 01:16:33.895019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-e0222d80-88b3-4eb1-9e12-37627b033f5e req-36163e22-778e-4547-b794-a51823da46e1 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048682 Jul 31 01:16:33.896342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 378/764] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:33.938902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:33.953126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4f0abd16-5de2-453d-a6a1-3297f7ca049c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.288931 Jul 31 01:16:33.954203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 387/765] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:33 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:34.168829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:34.170051 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:34.189688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:34.190606 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:34.192261 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:34.193195 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:34.690629 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:34.703164 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3a6cdbd5-ecd7-4457-864e-1c650987dd35 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1509 microversion: 2.1 time: 0.544134 Jul 31 01:16:34.704179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 379/766] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:34 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1509 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:34.978166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:34.979043 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:34.987580 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:34.988635 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:34.989568 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:34.990082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:35.053848 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-1bfd38b7-ae56-46ce-8eed-c2c77d3062f4 req-6f58720d-7bb6-4cb7-9f70-4171e03f1e66 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "973d94b6-5c7b-4acf-a633-8ed031945845", "name": "network-vif-deleted", "tag": "05db22e9-ff42-4276-93e4-b10ebcebcb40"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:35.085864 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1bfd38b7-ae56-46ce-8eed-c2c77d3062f4 req-6f58720d-7bb6-4cb7-9f70-4171e03f1e66 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:35.087044 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1bfd38b7-ae56-46ce-8eed-c2c77d3062f4 req-6f58720d-7bb6-4cb7-9f70-4171e03f1e66 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:35.144601 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-1bfd38b7-ae56-46ce-8eed-c2c77d3062f4 req-6f58720d-7bb6-4cb7-9f70-4171e03f1e66 service nova] Creating event network-vif-deleted:05db22e9-ff42-4276-93e4-b10ebcebcb40 for instance 973d94b6-5c7b-4acf-a633-8ed031945845 on hv-cinder-maste Jul 31 01:16:35.156701 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-1bfd38b7-ae56-46ce-8eed-c2c77d3062f4 req-6f58720d-7bb6-4cb7-9f70-4171e03f1e66 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.114820 Jul 31 01:16:35.159052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 380/767] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:35.228648 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:35.240292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ba5e2341-ae9a-4d22-a80d-5b73bb8e8e61 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.266763 Jul 31 01:16:35.241553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 388/768] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:34 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:35.736682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:35.738496 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:35.748444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:35.749155 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:35.750804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:35.751977 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:35.818962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:35.820397 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:35.821233 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lazy-loading 'fault' on Instance uuid 973d94b6-5c7b-4acf-a633-8ed031945845 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:35.917002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:35.925822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-186afcb8-8894-4543-aaa8-c7cda6c6f544 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 200 len: 1312 microversion: 2.1 time: 0.198994 Jul 31 01:16:35.926709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 381/769] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:35 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 1312 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:36.124711 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-ff3bb387-d7d0-43a5-af75-f5ce9cf4a8e9 req-d3f5f611-0aa9-48a8-98d9-380f9eabf9a5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7a3b72f2-b29c-4115-b742-1fa4a86e06b6", "tag": "f704de2f-7bdb-4ef2-80b7-29cb6b9472eb"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:36.125720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:36.126446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:36.139063 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.139512 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ff3bb387-d7d0-43a5-af75-f5ce9cf4a8e9 req-d3f5f611-0aa9-48a8-98d9-380f9eabf9a5 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.139982 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.140380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ff3bb387-d7d0-43a5-af75-f5ce9cf4a8e9 req-d3f5f611-0aa9-48a8-98d9-380f9eabf9a5 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.141571 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.142518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.158136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-ff3bb387-d7d0-43a5-af75-f5ce9cf4a8e9 req-d3f5f611-0aa9-48a8-98d9-380f9eabf9a5 service nova] Creating event network-changed:f704de2f-7bdb-4ef2-80b7-29cb6b9472eb for instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 on hv-cinder-maste Jul 31 01:16:36.164559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-ff3bb387-d7d0-43a5-af75-f5ce9cf4a8e9 req-d3f5f611-0aa9-48a8-98d9-380f9eabf9a5 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044878 Jul 31 01:16:36.166212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 382/770] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:36.288464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:36.289812 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:36.296626 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lazy-loading 'services' on Instance uuid 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:36.304361 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.305403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.306893 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.307739 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.431576 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-95e11100-7269-4c19-aad4-b1ab332744a2 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "DELETE /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments/79494aed-a098-435e-80ac-698a6aa289b4" status: 202 len: 0 microversion: 2.1 time: 0.314651 Jul 31 01:16:36.433334 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 389/771] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:16:36 2022] DELETE /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments/79494aed-a098-435e-80ac-698a6aa289b4 => generated 0 bytes in 318 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:36.453355 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4a708e42-9984-47bf-9c4b-9c7aeb901e97 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:36.460325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4a708e42-9984-47bf-9c4b-9c7aeb901e97 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:36.473793 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a708e42-9984-47bf-9c4b-9c7aeb901e97 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.475107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a708e42-9984-47bf-9c4b-9c7aeb901e97 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.477042 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a708e42-9984-47bf-9c4b-9c7aeb901e97 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.478015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a708e42-9984-47bf-9c4b-9c7aeb901e97 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.579245 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4a708e42-9984-47bf-9c4b-9c7aeb901e97 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.136784 Jul 31 01:16:36.587570 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 390/772] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:36 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments => generated 197 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:36.728740 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:36.741841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dff1435c-b7da-4934-aff8-44b00fa6736b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.468335 Jul 31 01:16:36.742914 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 383/773] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:36 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:36.962148 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:36.962926 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: 973d94b6-5c7b-4acf-a633-8ed031945845] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:36.970653 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.971248 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.972209 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:36.972751 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:36.991001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] HTTP exception thrown: Instance 973d94b6-5c7b-4acf-a633-8ed031945845 could not be found. Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Request method failure captured: Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 HTTP/1.1 Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dZ2ZANicX0K5Ht7oF4q37bz3ry4asTie3_vAAkdfXMm75OrKX_x8Trt62dwbXNsF8s_DZQ_uJMxtIx3O8383U61kM2V-WQSwIut0aag7RRA-YQK94v4Eloyyy8GU4XvoIDt7mCs5Ik6R1dpfM2P_EvHaOxL4bU--FT8De4fXyZLnrw Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:16:36.992920 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_3d6a76e846cb48bb91eab4da1c842f2b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 52e5a4deed99442e8eed62f334aa670d Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 973d94b6-5c7b-4acf-a633-8ed031945845 could not be found. Jul 31 01:16:36.996510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '973d94b6-5c7b-4acf-a633-8ed031945845'} Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 973d94b6-5c7b-4acf-a633-8ed031945845 could not be found. Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:36.998277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 973d94b6-5c7b-4acf-a633-8ed031945845 could not be found. Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 973d94b6-5c7b-4acf-a633-8ed031945845 could not be found. Jul 31 01:16:37.001399 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:37.004795 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Returning 404 to user: Instance 973d94b6-5c7b-4acf-a633-8ed031945845 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:37.004795 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-703b5d55-17fd-46fa-9cbb-2c70ea003569 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845" status: 404 len: 111 microversion: 2.1 time: 0.037458 Jul 31 01:16:37.004795 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 391/774] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:36 2022] GET /compute/v2.1/servers/973d94b6-5c7b-4acf-a633-8ed031945845 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:16:37.008380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:37.008976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:37.016542 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.017098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.017968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.018483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.048287 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:16:37.082891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:37.173801 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7f780942-e032-433a-b020-63ebd0947234 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 204 len: 0 microversion: 2.1 time: 0.169300 Jul 31 01:16:37.175221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 384/775] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:16:37 2022] DELETE /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:16:37.188819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:37.189950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:37.207946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.208396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.209708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.210438 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.433331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:37.444837 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5c5571de-b68e-4eec-8ce8-2aa92aaa895d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.263135 Jul 31 01:16:37.445719 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 392/776] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:37 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:37.603589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ea234e6b-563c-4086-aab9-ce5eed825949 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:37.608547 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ea234e6b-563c-4086-aab9-ce5eed825949 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:37.621700 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea234e6b-563c-4086-aab9-ce5eed825949 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.622480 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea234e6b-563c-4086-aab9-ce5eed825949 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.623902 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea234e6b-563c-4086-aab9-ce5eed825949 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.624966 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ea234e6b-563c-4086-aab9-ce5eed825949 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.686876 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ea234e6b-563c-4086-aab9-ce5eed825949 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.093557 Jul 31 01:16:37.687517 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 385/777] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:37 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments => generated 197 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:37.766096 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:37.766733 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:37.773285 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.773819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.774549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:37.774991 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:37.955266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:37.966122 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d132f6c2-b890-467d-a6c3-1a52591c9720 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.204371 Jul 31 01:16:37.966726 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 393/778] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:37 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:38.468469 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:38.469067 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:38.476645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:38.477175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:38.478085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:38.478702 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:38.627421 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:38.635671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-61d84f5d-f5a8-4557-9b64-78e2df6202e8 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.171550 Jul 31 01:16:38.636780 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 386/779] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:38 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:38.702042 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8cad92f3-9335-437a-a01a-12c8cca86515 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:38.704537 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8cad92f3-9335-437a-a01a-12c8cca86515 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:38.713477 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cad92f3-9335-437a-a01a-12c8cca86515 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:38.714140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cad92f3-9335-437a-a01a-12c8cca86515 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:38.715833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cad92f3-9335-437a-a01a-12c8cca86515 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:38.716514 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cad92f3-9335-437a-a01a-12c8cca86515 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:38.750573 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8cad92f3-9335-437a-a01a-12c8cca86515 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053779 Jul 31 01:16:38.751101 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 394/780] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:38 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:38.988527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:38.989185 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:38.999237 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:38.999752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:39.000560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:39.001057 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:39.152806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:39.160844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-312d65f9-157f-40f0-bb22-304b696c756d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.176288 Jul 31 01:16:39.161439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 387/781] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:38 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:39.657684 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:39.658300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:39.665317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:39.665764 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:39.666620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:39.667054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:39.770007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b76465d8-c9f7-4178-9a4b-7458103b80f9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:39.779097 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b76465d8-c9f7-4178-9a4b-7458103b80f9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:39.799158 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b76465d8-c9f7-4178-9a4b-7458103b80f9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:39.800091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b76465d8-c9f7-4178-9a4b-7458103b80f9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:39.801811 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b76465d8-c9f7-4178-9a4b-7458103b80f9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:39.802841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b76465d8-c9f7-4178-9a4b-7458103b80f9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:39.846573 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b76465d8-c9f7-4178-9a4b-7458103b80f9 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087273 Jul 31 01:16:39.852043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 388/782] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:39 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:39.889314 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:39.900476 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:39.904727 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2753c775-e394-4a0a-9c70-420eaa22743d tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1523 microversion: 2.1 time: 0.250980 Jul 31 01:16:39.905656 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 395/783] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:39 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1523 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:40.184361 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:40.186276 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:40.196231 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:40.196767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:40.197995 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:40.204347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:40.505656 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:40.526054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e624df72-949e-4c7c-b81d-0c19edad7146 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.347156 Jul 31 01:16:40.527112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 389/784] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:40 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:40.838572 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-0842a885-b718-4698-85bc-c3faf2c94ce4 req-7fb1d194-c608-4c53-b5e1-6a60806b6d4c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d2206492-6f7a-4a00-ac23-63200f66c940", "name": "network-vif-deleted", "tag": "78c48998-1895-47eb-bd4c-6c4a5145f1e5"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:40.853780 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-0842a885-b718-4698-85bc-c3faf2c94ce4 req-7fb1d194-c608-4c53-b5e1-6a60806b6d4c service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:40.866342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ef677d03-9f34-432e-ace1-7fafbe0f63d1 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:40.871135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ef677d03-9f34-432e-ace1-7fafbe0f63d1 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:40.872423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-0842a885-b718-4698-85bc-c3faf2c94ce4 req-7fb1d194-c608-4c53-b5e1-6a60806b6d4c service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:40.906410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef677d03-9f34-432e-ace1-7fafbe0f63d1 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:40.906869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef677d03-9f34-432e-ace1-7fafbe0f63d1 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:40.907955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef677d03-9f34-432e-ace1-7fafbe0f63d1 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:40.908596 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef677d03-9f34-432e-ace1-7fafbe0f63d1 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:40.930699 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-0842a885-b718-4698-85bc-c3faf2c94ce4 req-7fb1d194-c608-4c53-b5e1-6a60806b6d4c service nova] Creating event network-vif-deleted:78c48998-1895-47eb-bd4c-6c4a5145f1e5 for instance d2206492-6f7a-4a00-ac23-63200f66c940 on hv-cinder-maste Jul 31 01:16:40.945365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-0842a885-b718-4698-85bc-c3faf2c94ce4 req-7fb1d194-c608-4c53-b5e1-6a60806b6d4c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.113075 Jul 31 01:16:40.947592 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 396/785] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:40.962070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:40.963319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:40.977622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ef677d03-9f34-432e-ace1-7fafbe0f63d1 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.118503 Jul 31 01:16:40.987962 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:40.989191 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 390/786] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:16:40 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments => generated 25 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:16:40.991255 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:40.996974 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:40.997472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:40.998477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:41.000611 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:41.016320 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:41.016963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:41.019569 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:41.020742 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:41.079820 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:41.080721 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:41.082435 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lazy-loading 'fault' on Instance uuid d2206492-6f7a-4a00-ac23-63200f66c940 {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:41.152527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] HTTP exception thrown: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 is not attached to volume 79494aed-a098-435e-80ac-698a6aa289b4 Jul 31 01:16:41.152857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52e5a4deed99442e8eed62f334aa670d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d6a76e846cb48bb91eab4da1c842f2b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Request method failure captured: Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments/79494aed-a098-435e-80ac-698a6aa289b4 HTTP/1.1 Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dbqOqHQ389pwrV35wBA8LDlDUPxbGTf9VpEzQk19rPSraAb85M9P4rsqDzlPL1FNMz38UHc-iUGGr7TDqV4D1Tqha-bjGuoTAXMxsErLJwZWwBzBvvP4PwfjU8tNX1urRtNzonbD2q_boTQN_5E2Of8BhIkDqQbdSewJR8Zug1Jp1Y Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: b018ba41816049feaa75e10489ec5281 Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-TestMinimumBasicScenario-536044842 Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:16:41.155962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b018ba41816049feaa75e10489ec5281"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b018ba41816049feaa75e10489ec5281"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b018ba41816049feaa75e10489ec5281"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b018ba41816049feaa75e10489ec5281", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-TestMinimumBasicScenario-536044842 Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: b018ba41816049feaa75e10489ec5281 Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-TestMinimumBasicScenario-536044842 Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-TestMinimumBasicScenario-536044842-project Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 1776c123f98c44b69807ec2b0281b718 Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-TestMinimumBasicScenario-536044842-project Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:16:41.158594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 is not attached to volume 79494aed-a098-435e-80ac-698a6aa289b4 Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'server_id': '7a3b72f2-b29c-4115-b742-1fa4a86e06b6', 'id': '79494aed-a098-435e-80ac-698a6aa289b4'} Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 79494aed-a098-435e-80ac-698a6aa289b4. Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:41.159967 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 is not attached to volume 79494aed-a098-435e-80ac-698a6aa289b4 Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 is not attached to volume 79494aed-a098-435e-80ac-698a6aa289b4 Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:41.161769 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Returning 404 to user: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 is not attached to volume 79494aed-a098-435e-80ac-698a6aa289b4 {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:41.162901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-abdd33b0-c9d3-4da3-ab0a-cc3d55d78cec tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "DELETE /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments/79494aed-a098-435e-80ac-698a6aa289b4" status: 404 len: 154 microversion: 2.1 time: 0.167320 Jul 31 01:16:41.162901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 391/787] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:16:40 2022] DELETE /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6/os-volume_attachments/79494aed-a098-435e-80ac-698a6aa289b4 => generated 154 bytes in 169 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:16:41.166577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4406a8a6-137a-4dfd-982b-fca51c976b3b tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 200 len: 1312 microversion: 2.1 time: 0.216997 Jul 31 01:16:41.171576 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 397/788] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:40 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 1312 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:41.563938 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:41.565055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:41.580080 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:41.580897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:41.583656 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:41.584443 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:41.778641 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:41.795075 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-94557bf4-f575-4673-ae37-af02593e3f81 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.238929 Jul 31 01:16:41.796165 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 392/789] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:41 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:42.202252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:42.203279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] [instance: d2206492-6f7a-4a00-ac23-63200f66c940] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:42.212019 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.212634 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:42.213515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.214084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:42.230975 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] HTTP exception thrown: Instance d2206492-6f7a-4a00-ac23-63200f66c940 could not be found. Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Request method failure captured: Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 HTTP/1.1 Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dZ2ZANicX0K5Ht7oF4q37bz3ry4asTie3_vAAkdfXMm75OrKX_x8Trt62dwbXNsF8s_DZQ_uJMxtIx3O8383U61kM2V-WQSwIut0aag7RRA-YQK94v4Eloyyy8GU4XvoIDt7mCs5Ik6R1dpfM2P_EvHaOxL4bU--FT8De4fXyZLnrw Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:16:42.232853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/3d6a76e846cb48bb91eab4da1c842f2b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_3d6a76e846cb48bb91eab4da1c842f2b", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 3d6a76e846cb48bb91eab4da1c842f2b Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-503554371 Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 52e5a4deed99442e8eed62f334aa670d Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-ServerRescueNegativeTestJSON-503554371-project Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance d2206492-6f7a-4a00-ac23-63200f66c940 could not be found. Jul 31 01:16:42.235384 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'd2206492-6f7a-4a00-ac23-63200f66c940'} Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d2206492-6f7a-4a00-ac23-63200f66c940 could not be found. Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:42.236802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d2206492-6f7a-4a00-ac23-63200f66c940 could not be found. Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d2206492-6f7a-4a00-ac23-63200f66c940 could not be found. Jul 31 01:16:42.239080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:42.241228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] Returning 404 to user: Instance d2206492-6f7a-4a00-ac23-63200f66c940 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:42.241228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-77d64bee-7327-4033-9234-b3c3086f7bf6 tempest-ServerRescueNegativeTestJSON-503554371 tempest-ServerRescueNegativeTestJSON-503554371-project] 10.222.0.42 "GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940" status: 404 len: 111 microversion: 2.1 time: 0.037114 Jul 31 01:16:42.241228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 398/790] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:42 2022] GET /compute/v2.1/servers/d2206492-6f7a-4a00-ac23-63200f66c940 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:16:42.346937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:42.347836 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:42.356535 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.357114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:42.358114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.358663 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:42.391670 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:16:42.511265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d644b197-2a1a-4892-97f0-2c174bc9ba5e tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "DELETE /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 204 len: 0 microversion: 2.1 time: 0.167684 Jul 31 01:16:42.513002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 393/791] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:16:42 2022] DELETE /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:16:42.524891 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:42.525978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:42.536299 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.536750 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:42.537545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.537980 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:42.832338 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:42.832711 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:42.841225 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.842206 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:42.844091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:42.845319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:43.041001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:43.063885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a1f57e8c-f041-4334-8257-64cb213b67aa tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1553 microversion: 2.1 time: 0.545605 Jul 31 01:16:43.065361 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 399/792] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:42 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1553 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:43.083252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:43.095934 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d1fb19aa-799d-4eeb-b0fc-4af5d169210f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.268723 Jul 31 01:16:43.096716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 394/793] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:42 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:44.089797 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:44.091925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:44.105644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:44.106414 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:44.107892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:44.108473 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:44.121836 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:44.124013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:44.144136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:44.145870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:44.147787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:44.148592 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:44.394907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:44.414095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e1e77b92-c0ff-4e17-9850-2c1b5a461b0c tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1553 microversion: 2.1 time: 0.330710 Jul 31 01:16:44.415137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 400/794] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:44 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1553 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:44.416764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:44.432413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-58c3b187-bb01-4d89-ab17-454c2730fa41 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.315942 Jul 31 01:16:44.434958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 395/795] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:44 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:45.436467 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:45.437833 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:45.445591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:45.446466 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:45.447591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:45.448253 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:45.458088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:45.459723 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:45.468095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:45.468970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:45.470156 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:45.470829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:45.717198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:45.732794 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b655f7cb-ee24-4e77-9fe8-1efdf8c1c508 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.279545 Jul 31 01:16:45.751619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 396/796] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:45 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:45.967981 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:46.007306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-057f8cf6-8c67-4f85-b54c-fce8c2aded33 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1553 microversion: 2.1 time: 0.574428 Jul 31 01:16:46.008184 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 401/797] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:45 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1553 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:46.066603 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-ff2592ac-4d28-4fda-ae57-fbeec43c196f req-7e5ac1d6-d627-45d7-9b8b-1b1229ab7e91 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7a3b72f2-b29c-4115-b742-1fa4a86e06b6", "name": "network-vif-deleted", "tag": "f704de2f-7bdb-4ef2-80b7-29cb6b9472eb"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:46.088209 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ff2592ac-4d28-4fda-ae57-fbeec43c196f req-7e5ac1d6-d627-45d7-9b8b-1b1229ab7e91 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:46.089725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-ff2592ac-4d28-4fda-ae57-fbeec43c196f req-7e5ac1d6-d627-45d7-9b8b-1b1229ab7e91 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:46.112783 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-ff2592ac-4d28-4fda-ae57-fbeec43c196f req-7e5ac1d6-d627-45d7-9b8b-1b1229ab7e91 service nova] Creating event network-vif-deleted:f704de2f-7bdb-4ef2-80b7-29cb6b9472eb for instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 on hv-cinder-maste Jul 31 01:16:46.122268 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-ff2592ac-4d28-4fda-ae57-fbeec43c196f req-7e5ac1d6-d627-45d7-9b8b-1b1229ab7e91 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062032 Jul 31 01:16:46.123784 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 397/798] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:46.764679 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:46.765898 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:46.776041 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:46.776669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:46.777583 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:46.778186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:46.989789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:47.009902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8390798b-089a-40ae-98b3-df99a683c13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.252965 Jul 31 01:16:47.010938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 402/799] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:46 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:47.052371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:47.053506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:47.064623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:47.065369 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:47.067097 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:47.068212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:47.244205 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1776c123f98c44b69807ec2b0281b718', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b018ba41816049feaa75e10489ec5281', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:47.267189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-73dfcb84-01e8-4f0a-a331-65750f939a7f tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 200 len: 1342 microversion: 2.1 time: 0.226854 Jul 31 01:16:47.269472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 398/800] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:47 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 1342 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:48.057490 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:48.059335 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:48.071977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:48.073109 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:48.075091 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:48.075918 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:48.310148 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:48.312809 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] [instance: 7a3b72f2-b29c-4115-b742-1fa4a86e06b6] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:48.321356 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:48.322482 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:48.325036 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:48.325901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:48.342510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:48.343260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] HTTP exception thrown: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 could not be found. Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Request method failure captured: Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 HTTP/1.1 Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dbqOqHQ389pwrV35wBA8LDlDUPxbGTf9VpEzQk19rPSraAb85M9P4rsqDzlPL1FNMz38UHc-iUGGr7TDqV4D1Tqha-bjGuoTAXMxsErLJwZWwBzBvvP4PwfjU8tNX1urRtNzonbD2q_boTQN_5E2Of8BhIkDqQbdSewJR8Zug1Jp1Y Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: b018ba41816049feaa75e10489ec5281 Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-TestMinimumBasicScenario-536044842 Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:16:48.344976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b018ba41816049feaa75e10489ec5281"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b018ba41816049feaa75e10489ec5281"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b018ba41816049feaa75e10489ec5281"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b018ba41816049feaa75e10489ec5281", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-TestMinimumBasicScenario-536044842 Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: b018ba41816049feaa75e10489ec5281 Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-TestMinimumBasicScenario-536044842 Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-TestMinimumBasicScenario-536044842-project Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 1776c123f98c44b69807ec2b0281b718 Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-TestMinimumBasicScenario-536044842-project Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 could not be found. Jul 31 01:16:48.347676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '7a3b72f2-b29c-4115-b742-1fa4a86e06b6'} Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 could not be found. Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:48.349171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 could not be found. Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 could not be found. Jul 31 01:16:48.353882 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:16:48.358534 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Returning 404 to user: Instance 7a3b72f2-b29c-4115-b742-1fa4a86e06b6 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:48.358534 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a861d1c7-4cdb-4063-8c2c-cdf63721e5a7 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6" status: 404 len: 111 microversion: 2.1 time: 0.043340 Jul 31 01:16:48.358534 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 399/801] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:48 2022] GET /compute/v2.1/servers/7a3b72f2-b29c-4115-b742-1fa4a86e06b6 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:16:48.361627 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-147eb244-5cb4-4c75-b533-b3bb0151e689 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] Calling method '.version_select at 0x7f3e7b70ba60>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:48.363200 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8c1f881b-77e0-4f83-825d-af8ac41e8623 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.313633 Jul 31 01:16:48.371416 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 403/802] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:48 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:48.399473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-147eb244-5cb4-4c75-b533-b3bb0151e689 tempest-TestMinimumBasicScenario-536044842 tempest-TestMinimumBasicScenario-536044842-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-2026137047" status: 202 len: 0 microversion: 2.1 time: 0.045089 Jul 31 01:16:48.400293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 400/803] 10.222.0.42 () {58 vars in 1272 bytes} [Sun Jul 31 01:16:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-2026137047 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 31 01:16:48.419244 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-37083aa9-e6b3-4477-a7ae-c335415e7e98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:48.420119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-37083aa9-e6b3-4477-a7ae-c335415e7e98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:48.435447 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37083aa9-e6b3-4477-a7ae-c335415e7e98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:48.436221 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37083aa9-e6b3-4477-a7ae-c335415e7e98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:48.437835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37083aa9-e6b3-4477-a7ae-c335415e7e98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:48.438481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-37083aa9-e6b3-4477-a7ae-c335415e7e98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:48.471029 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-37083aa9-e6b3-4477-a7ae-c335415e7e98 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/ips" status: 200 len: 153 microversion: 2.20 time: 0.060736 Jul 31 01:16:48.472311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 404/804] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Jul 31 01:16:48 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/ips => generated 153 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 31 01:16:49.064856 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-bbf8694a-a8f4-473e-8729-e12375991b34 req-accacb2c-8ae0-460e-bbc6-aa08058d38d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cf3658-0cb6-48b8-b152-72b915385fbb", "name": "network-vif-unplugged", "status": "completed", "tag": "38f41703-e2b2-4f28-9dc2-9118c346cfba"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:49.095190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-bbf8694a-a8f4-473e-8729-e12375991b34 req-accacb2c-8ae0-460e-bbc6-aa08058d38d7 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:49.105643 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-bbf8694a-a8f4-473e-8729-e12375991b34 req-accacb2c-8ae0-460e-bbc6-aa08058d38d7 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:49.125794 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=96,nova_cell1:INSERT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:49.153133 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.compute.server_external_events [req-bbf8694a-a8f4-473e-8729-e12375991b34 req-accacb2c-8ae0-460e-bbc6-aa08058d38d7 service nova] Unable to find a host for instance 69cf3658-0cb6-48b8-b152-72b915385fbb. Dropping event network-vif-unplugged {{(pid=117942) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 31 01:16:49.155520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-bbf8694a-a8f4-473e-8729-e12375991b34 req-accacb2c-8ae0-460e-bbc6-aa08058d38d7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.094898 Jul 31 01:16:49.157813 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 401/805] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:16:49 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 98 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Jul 31 01:16:49.403642 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:49.405022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:49.422949 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:49.423796 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:49.425009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:49.425769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:49.665591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:49.680297 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8136872a-f594-48b2-85f4-62f2c3710f9c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1511 microversion: 2.1 time: 0.283634 Jul 31 01:16:49.681280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 405/806] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:49 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1511 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:49.719871 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:49.720954 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:49.734521 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:49.735406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:49.736696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:49.737463 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:49.821752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=117942) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Jul 31 01:16:49.967171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8f562405-3da8-43f0-8075-a2a8ead79769 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "POST /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb/action" status: 202 len: 0 microversion: 2.1 time: 0.255024 Jul 31 01:16:49.974451 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 402/807] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:16:49 2022] POST /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb/action => generated 0 bytes in 263 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:49.988078 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:49.989602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:50.004100 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:50.005461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:50.007505 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:50.008875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:50.279056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-22e911ac-4b0e-4a95-8615-36489daff066 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Action: 'create', calling method: .version_select at 0x7f3e7b6e6820>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1160302475"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:50.364669 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:50.378681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-40ea48ba-fb04-4995-8fd8-73560b732138 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1519 microversion: 2.1 time: 0.401083 Jul 31 01:16:50.391831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 403/808] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:49 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1519 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:50.894628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-22e911ac-4b0e-4a95-8615-36489daff066 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.928256 Jul 31 01:16:50.896608 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 406/809] 10.222.0.42 () {60 vars in 1155 bytes} [Sun Jul 31 01:16:49 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 931 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 31 01:16:50.919872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-efe8a914-7f3c-41f4-85cb-226d7774a415 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b636430>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:51.027412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-efe8a914-7f3c-41f4-85cb-226d7774a415 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445" status: 200 len: 345 microversion: 2.1 time: 0.114877 Jul 31 01:16:51.028984 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 404/810] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:16:50 2022] GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 => generated 345 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:51.254907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:51.258387 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:51.273810 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:51.275105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:51.276987 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:51.277780 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:51.334127 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lazy-loading 'flavor' on Instance uuid c03b1463-390f-430b-9bdf-64322c030504 {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:16:51.409290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:51.413873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:51.435493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:51.437958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:51.442868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:51.445520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:51.632534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c8069397-475a-48b4-a026-b779aabea590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/action" status: 202 len: 0 microversion: 2.20 time: 0.386235 Jul 31 01:16:51.633171 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 407/811] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Jul 31 01:16:51 2022] POST /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/action => generated 0 bytes in 389 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:16:51.644255 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:51.645301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:51.652727 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:51.653529 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:51.654575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:51.655253 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:51.672206 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:51.694479 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8716463c-a89f-49f9-b103-a023675fb8f3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1519 microversion: 2.1 time: 0.294824 Jul 31 01:16:51.696325 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 405/812] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:51 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1519 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:51.994689 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:52.005485 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:52.007441 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:52.009138 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-60def287-ed7b-42c1-ad34-76dd572f914a tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.369656 Jul 31 01:16:52.009765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 408/813] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:16:51 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:16:52.042764 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a1e85ddb-7d1a-4fc0-96d1-b8e3f6af0063 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b636820>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:52.121939 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a1e85ddb-7d1a-4fc0-96d1-b8e3f6af0063 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445" status: 200 len: 346 microversion: 2.1 time: 0.083282 Jul 31 01:16:52.122682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 406/814] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:16:52 2022] GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 => generated 346 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:52.143616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d52f5bd5-0931-48c5-9a48-1eea1cbca24c tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Action: 'create', calling method: .version_select at 0x7f3e7b6aa040>, body: {"snapshot": {"volume_id": "6804ca2c-dede-4c88-b74a-4e9251bcc445", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-698079898"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:16:52.434113 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d52f5bd5-0931-48c5-9a48-1eea1cbca24c tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.300701 Jul 31 01:16:52.435695 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 409/815] 10.222.0.42 () {60 vars in 1162 bytes} [Sun Jul 31 01:16:52 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:52.447020 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2a843429-55af-41fb-922a-e512ac2a5d82 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b619ee0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:52.487387 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2a843429-55af-41fb-922a-e512ac2a5d82 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 286 microversion: 2.1 time: 0.044285 Jul 31 01:16:52.488272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 407/816] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:52 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 286 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:52.507814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e8067572-81cc-4a9b-a210-b0b7ff8202b9 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b6499d0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:52.544102 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e8067572-81cc-4a9b-a210-b0b7ff8202b9 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.040712 Jul 31 01:16:52.544914 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 410/817] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Jul 31 01:16:52 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:52.558976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9794f4bd-e43f-4c19-9aba-e9c75c8323a4 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b5b0940>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:52.598973 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9794f4bd-e43f-4c19-9aba-e9c75c8323a4 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 286 microversion: 2.1 time: 0.042114 Jul 31 01:16:52.602210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 408/818] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:52 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 286 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:52.718528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:52.719153 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:52.725867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:52.726329 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:52.727141 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:52.727640 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:52.894735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:52.904187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cdd06251-8e47-4944-ac66-af0468593e92 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.190022 Jul 31 01:16:52.907410 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 411/819] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:52 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:53.040201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:53.041482 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:53.055711 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:53.056776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:53.058522 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:53.059418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:53.418001 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:53.431951 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:53.435245 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:53.437135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-028754a6-f784-4959-97ad-30319632b1a4 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.404998 Jul 31 01:16:53.437960 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 409/820] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:16:53 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:16:53.615800 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f85d98b9-d83d-4323-a73d-1bd12d501ddb tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b791430>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:53.650925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f85d98b9-d83d-4323-a73d-1bd12d501ddb tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 286 microversion: 2.1 time: 0.039902 Jul 31 01:16:53.652142 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 412/821] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:53 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 286 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:53.944438 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:53.945645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:53.971402 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:53.972818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:53.977082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:53.977652 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:54.242292 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:54.252470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a6edfe38-ab94-4a54-911e-8889e8429fc3 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.314330 Jul 31 01:16:54.253147 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 410/822] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:53 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:16:54.468959 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:54.470825 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:54.484440 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:54.485454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:54.487004 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:54.487910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:54.668528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8e27659f-2994-439d-8e6f-1964dec159ad tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e8007e4c0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:54.680961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:54.696724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:54.700715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:54.703503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-48da624d-a81b-45fd-abbe-eb204d719177 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.242937 Jul 31 01:16:54.711675 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8e27659f-2994-439d-8e6f-1964dec159ad tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 286 microversion: 2.1 time: 0.050076 Jul 31 01:16:54.712289 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 411/823] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:54 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 286 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:16:54.716233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 413/824] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:16:54 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:16:55.286107 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:55.287139 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:55.290618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=2,nova_api:UPDATE=4 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:55.298336 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:55.298977 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:55.300081 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:55.300650 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:55.530621 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:55.540250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f035b02d-e851-4330-a3f5-1bf942f392b5 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.259224 Jul 31 01:16:55.540870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 412/825] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:55 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:55.728463 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-23e68323-3590-481d-b852-b3c0a0156ce6 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b6d4dc0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:55.733150 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:55.733886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:55.745528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:55.746117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:55.747179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:55.748032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:55.803587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-23e68323-3590-481d-b852-b3c0a0156ce6 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 286 microversion: 2.1 time: 0.081532 Jul 31 01:16:55.807879 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 414/826] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:55 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 286 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:55.966870 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:55.984103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:55.988924 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:55.991395 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8e46dcb5-b7c5-400e-9118-057ad0aff3cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.262018 Jul 31 01:16:55.992547 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 413/827] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:16:55 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:16:56.575851 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:56.576578 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:56.578808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=64 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:56.589200 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:56.589882 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:56.591167 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:56.591644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:56.820360 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-64cd1ab1-83b4-45a1-9ed2-45aa5a08e6c4 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b5bfb80>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:56.866525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-64cd1ab1-83b4-45a1-9ed2-45aa5a08e6c4 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 286 microversion: 2.1 time: 0.052072 Jul 31 01:16:56.867582 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 414/828] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:56 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 286 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:56.987884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:57.004637 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2430dcc7-b640-45a2-93a7-9fbc385dc475 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.432891 Jul 31 01:16:57.006328 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 415/829] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:56 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:57.030414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:57.031421 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:57.039316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:57.040113 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:57.041146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:57.041611 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:57.202141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:57.214252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:57.216153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:57.217413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a369c2ed-cbbb-4b90-b6fa-1c1bea82cf44 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.192277 Jul 31 01:16:57.217998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 415/830] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:16:57 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:16:57.886486 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-160a3f99-7c1b-4268-b80d-760142a3ca26 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b6aac10>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:57.930364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-160a3f99-7c1b-4268-b80d-760142a3ca26 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 287 microversion: 2.1 time: 0.051471 Jul 31 01:16:57.935755 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 416/831] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:57 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 287 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:57.947015 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f9b9f4d9-58a4-4755-b091-3d729bd58a32 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b5a7ca0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:58.035131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:58.036063 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:58.066633 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:58.067534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:58.071180 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:58.072000 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:58.074519 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f9b9f4d9-58a4-4755-b091-3d729bd58a32 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "DELETE /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 202 len: 0 microversion: 2.1 time: 0.135397 Jul 31 01:16:58.076820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 416/832] 10.222.0.42 () {58 vars in 1254 bytes} [Sun Jul 31 01:16:57 2022] DELETE /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:58.093660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-97b9d100-2e72-408b-96b4-d255adf74862 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b5bf700>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:58.221408 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-97b9d100-2e72-408b-96b4-d255adf74862 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 200 len: 286 microversion: 2.1 time: 0.137804 Jul 31 01:16:58.222453 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 417/833] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:58 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:58.242194 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:58.243908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:58.264742 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:58.265409 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:58.266578 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:58.267319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:58.390471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:58.394160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:16:58.405075 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-914d5c95-3c5c-4d1c-89e0-f4c638eb6bca tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.379827 Jul 31 01:16:58.405680 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 417/834] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:58 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:16:58.621216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:58.639106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:58.641598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:58.643232 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-202113e7-2227-4f39-9c8a-1ba840ad2855 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.405101 Jul 31 01:16:58.644400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 418/835] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:16:58 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:16:59.242097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-15a9512a-6e02-4c6c-9876-df9d10dc7c62 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b7cdaf0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:59.297321 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-15a9512a-6e02-4c6c-9876-df9d10dc7c62 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] HTTP exception thrown: Snapshot 373dedb2-b9c4-41ba-a90b-3a52f4f6c779 could not be found. Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-15a9512a-6e02-4c6c-9876-df9d10dc7c62 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Request method failure captured: Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 HTTP/1.1 Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dgBqIHMSD2A_1IlPhutgMMxcv-aRWYWFuz0r4zaht5TLAr0OOhFtrTZnXAnz_fYDzhu9jjaEdQ6HQb1xyjNU64TAtguuNwO6lTmQ9fcBt1aMjhcWni4i1FMfVMrnsIRh364Sr4KEw_3xRb--JC0AFRZyIkNNpuM2boIBDJWRihWjAY Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:16:59.300877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2dbb3f502e3f4eb591d8ca4588efa00f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 82789ff85af6474b8db657cf4c71f6f2 Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Snapshot 373dedb2-b9c4-41ba-a90b-3a52f4f6c779 could not be found. Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '373dedb2-b9c4-41ba-a90b-3a52f4f6c779'} Jul 31 01:16:59.304046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 373dedb2-b9c4-41ba-a90b-3a52f4f6c779 could not be found. (HTTP 404) (Request-ID: req-4622edd2-3145-47c8-998a-016d3ed57c18) Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:59.305606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:16:59.308906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 373dedb2-b9c4-41ba-a90b-3a52f4f6c779 could not be found. Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jul 31 01:16:59.317777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 373dedb2-b9c4-41ba-a90b-3a52f4f6c779 could not be found. Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 373dedb2-b9c4-41ba-a90b-3a52f4f6c779 could not be found. Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-15a9512a-6e02-4c6c-9876-df9d10dc7c62 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Returning 404 to user: Snapshot 373dedb2-b9c4-41ba-a90b-3a52f4f6c779 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:16:59.322331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-15a9512a-6e02-4c6c-9876-df9d10dc7c62 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779" status: 404 len: 111 microversion: 2.1 time: 0.079650 Jul 31 01:16:59.325125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 418/836] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Jul 31 01:16:59 2022] GET /compute/v2.1/os-snapshots/373dedb2-b9c4-41ba-a90b-3a52f4f6c779 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:16:59.336055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-88ad3936-4cc3-4b9e-9d76-0412277f64e4 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b5bfaf0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:59.448015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:59.453201 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:59.479733 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-88ad3936-4cc3-4b9e-9d76-0412277f64e4 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445" status: 202 len: 0 microversion: 2.1 time: 0.157965 Jul 31 01:16:59.484112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 419/837] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:16:59 2022] DELETE /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 => generated 0 bytes in 164 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:16:59.487646 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:59.488854 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:59.490086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d6a829b5-a812-4146-ad31-41257a617903 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e8009f0d0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:59.490902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:59.492078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:59.613596 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d6a829b5-a812-4146-ad31-41257a617903 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445" status: 200 len: 345 microversion: 2.1 time: 0.127848 Jul 31 01:16:59.614543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 420/838] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:16:59 2022] GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 => generated 345 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:16:59.672510 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:16:59.673594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:16:59.682934 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:59.683590 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:59.684472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:16:59.685112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:16:59.866366 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:16:59.887970 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c21d434e-d85b-42c8-a2b7-520b8af97bdd tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.447211 Jul 31 01:16:59.889410 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 419/839] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:16:59 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:00.057032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:00.083623 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:00.091431 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:00.093773 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-02c0254a-76a1-45d3-a2e0-eeefc2249e17 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.425908 Jul 31 01:17:00.099622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 421/840] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:16:59 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:00.635972 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-94b21c6b-1d38-4406-8790-1082afaf1e52 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b6303a0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:00.698561 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-94b21c6b-1d38-4406-8790-1082afaf1e52 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] HTTP exception thrown: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-94b21c6b-1d38-4406-8790-1082afaf1e52 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Request method failure captured: Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 HTTP/1.1 Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dgBqIHMSD2A_1IlPhutgMMxcv-aRWYWFuz0r4zaht5TLAr0OOhFtrTZnXAnz_fYDzhu9jjaEdQ6HQb1xyjNU64TAtguuNwO6lTmQ9fcBt1aMjhcWni4i1FMfVMrnsIRh364Sr4KEw_3xRb--JC0AFRZyIkNNpuM2boIBDJWRihWjAY Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:17:00.705527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2dbb3f502e3f4eb591d8ca4588efa00f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 82789ff85af6474b8db657cf4c71f6f2 Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '6804ca2c-dede-4c88-b74a-4e9251bcc445'} Jul 31 01:17:00.708260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. (HTTP 404) (Request-ID: req-321d8b29-f981-4d70-8e0f-ea244907d0df) Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.718256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:17:00.721256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 31 01:17:00.724856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-94b21c6b-1d38-4406-8790-1082afaf1e52 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Returning 404 to user: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-94b21c6b-1d38-4406-8790-1082afaf1e52 tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445" status: 404 len: 109 microversion: 2.1 time: 0.081484 Jul 31 01:17:00.727735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 420/841] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:17:00 2022] GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 => generated 109 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:17:00.730553 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-89e2a67e-23ab-494a-998e-211608ec91de tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b5b0040>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:00.786535 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-89e2a67e-23ab-494a-998e-211608ec91de tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] HTTP exception thrown: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-89e2a67e-23ab-494a-998e-211608ec91de tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Request method failure captured: Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 HTTP/1.1 Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dgBqIHMSD2A_1IlPhutgMMxcv-aRWYWFuz0r4zaht5TLAr0OOhFtrTZnXAnz_fYDzhu9jjaEdQ6HQb1xyjNU64TAtguuNwO6lTmQ9fcBt1aMjhcWni4i1FMfVMrnsIRh364Sr4KEw_3xRb--JC0AFRZyIkNNpuM2boIBDJWRihWjAY Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:17:00.790901 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2dbb3f502e3f4eb591d8ca4588efa00f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 82789ff85af6474b8db657cf4c71f6f2 Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '6804ca2c-dede-4c88-b74a-4e9251bcc445'} Jul 31 01:17:00.794416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. (HTTP 404) (Request-ID: req-13fdb460-1268-41c2-80a6-faf6fed9cb2d) Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.796197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 31 01:17:00.798832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 31 01:17:00.804049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-89e2a67e-23ab-494a-998e-211608ec91de tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Returning 404 to user: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-89e2a67e-23ab-494a-998e-211608ec91de tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445" status: 404 len: 109 microversion: 2.1 time: 0.082090 Jul 31 01:17:00.806470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 422/842] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:17:00 2022] DELETE /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 => generated 109 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:17:00.813396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7c564ce3-db85-4794-8180-827fda8a284a tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Calling method '.version_select at 0x7f3e7b6515e0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:00.846238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-7c564ce3-db85-4794-8180-827fda8a284a tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] HTTP exception thrown: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7c564ce3-db85-4794-8180-827fda8a284a tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Request method failure captured: Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 HTTP/1.1 Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dgBqIHMSD2A_1IlPhutgMMxcv-aRWYWFuz0r4zaht5TLAr0OOhFtrTZnXAnz_fYDzhu9jjaEdQ6HQb1xyjNU64TAtguuNwO6lTmQ9fcBt1aMjhcWni4i1FMfVMrnsIRh364Sr4KEw_3xRb--JC0AFRZyIkNNpuM2boIBDJWRihWjAY Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:17:00.849327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2dbb3f502e3f4eb591d8ca4588efa00f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2dbb3f502e3f4eb591d8ca4588efa00f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 2dbb3f502e3f4eb591d8ca4588efa00f Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1473823654 Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 82789ff85af6474b8db657cf4c71f6f2 Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1473823654-project Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '6804ca2c-dede-4c88-b74a-4e9251bcc445'} Jul 31 01:17:00.850894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. (HTTP 404) (Request-ID: req-96a17c23-4a2b-459f-9029-a0b49c99e149) Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.851846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:17:00.852994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 31 01:17:00.855010 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. Jul 31 01:17:00.856186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:00.858896 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7c564ce3-db85-4794-8180-827fda8a284a tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] Returning 404 to user: Volume 6804ca2c-dede-4c88-b74a-4e9251bcc445 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:17:00.860506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7c564ce3-db85-4794-8180-827fda8a284a tempest-VolumesSnapshotsTestJSON-1473823654 tempest-VolumesSnapshotsTestJSON-1473823654-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445" status: 404 len: 109 microversion: 2.1 time: 0.051170 Jul 31 01:17:00.861166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 421/843] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:17:00 2022] GET /compute/v2.1/os-volumes/6804ca2c-dede-4c88-b74a-4e9251bcc445 => generated 109 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:17:00.915664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:00.918198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:00.932050 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:00.932876 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:00.934198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:00.934942 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:01.141149 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:01.142298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:01.167232 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:01.168432 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:01.170792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:01.171508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:01.306909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:01.320863 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-648da4bc-95af-4bfe-87f4-460e4fc5b13e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.413127 Jul 31 01:17:01.322065 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 423/844] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:00 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:01.462396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:01.478915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:01.482233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:01.484249 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-275975f5-b13e-410f-a285-ee70d35bc3a6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.353900 Jul 31 01:17:01.485195 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 422/845] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:01 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:02.355175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:02.356630 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:02.365617 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:02.366063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:02.368105 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:02.368866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:02.525589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:02.528524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:02.542906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:02.544338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:02.545906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:02.546854 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:02.603348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:02.614848 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6f14c921-c327-4242-8c00-9dc622114345 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.264684 Jul 31 01:17:02.616755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 424/846] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:02 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:02.795688 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:02.815624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:02.819764 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:02.822497 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6d513088-b18c-4711-84b3-5ae8a5493283 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.303700 Jul 31 01:17:02.824266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 423/847] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:02 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:03.601992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-16926cd6-9f96-4f0c-ae7e-70eb453b7feb tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Action: 'create', calling method: .version_select at 0x7f3e7b6365e0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-171303821", "metadata": {"Type": "work"}}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:03.656887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:03.657876 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:03.670089 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:03.670854 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:03.672823 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:03.673606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:03.866317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:03.886235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-17a79dc8-341b-4fcf-a9ea-3f52c237b970 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.235722 Jul 31 01:17:03.887723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 424/848] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:03 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:03.901342 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:03.902925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:03.916940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:03.918062 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:03.920411 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:03.921640 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:04.148649 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-16926cd6-9f96-4f0c-ae7e-70eb453b7feb tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.880726 Jul 31 01:17:04.151076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 425/849] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Jul 31 01:17:03 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 886 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Jul 31 01:17:04.175477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-272194a7-4d95-46cd-bb1c-21183aa76f94 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Calling method '.version_select at 0x7f3e7b5bfa60>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:04.224120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:04.239511 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:04.242723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:04.245382 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3fe8a478-1c34-488a-8e66-0ce3afab0cf3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.354922 Jul 31 01:17:04.246501 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 425/850] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:03 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:04.262238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-272194a7-4d95-46cd-bb1c-21183aa76f94 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c" status: 200 len: 352 microversion: 2.1 time: 0.095860 Jul 31 01:17:04.264444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 426/851] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:17:04 2022] GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c => generated 352 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:17:04.911533 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:04.912785 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:04.920633 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:04.921137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:04.921952 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:04.922394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:05.089630 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:05.098095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-fae3b805-ccc1-472c-bb1e-43b18008c596 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.191011 Jul 31 01:17:05.098800 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 426/852] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:04 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:05.285561 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:05.286222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:05.286950 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-07b40399-eef5-40f1-8d33-ea759f992efc tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Calling method '.version_select at 0x7f3e7b636790>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:05.304558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:05.305961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:05.306954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:05.307452 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:05.396375 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-07b40399-eef5-40f1-8d33-ea759f992efc tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c" status: 200 len: 353 microversion: 2.1 time: 0.116951 Jul 31 01:17:05.397948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 427/853] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:17:05 2022] GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c => generated 353 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:17:05.430434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-302c1d46-95dc-4fee-a0b1-a3b235e9d745 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Calling method '.version_select at 0x7f3e7b5a7ca0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:05.527541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-302c1d46-95dc-4fee-a0b1-a3b235e9d745 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c" status: 200 len: 353 microversion: 2.1 time: 0.113690 Jul 31 01:17:05.529360 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 428/854] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:17:05 2022] GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c => generated 353 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:17:05.555340 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5b08dcae-1e63-46c2-a127-acb4a6b86616 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Calling method '.version_select at 0x7f3e7b5b0160>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:05.705228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5b08dcae-1e63-46c2-a127-acb4a6b86616 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c" status: 202 len: 0 microversion: 2.1 time: 0.161735 Jul 31 01:17:05.705824 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 429/855] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:17:05 2022] DELETE /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:17:05.719055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-33c8c5b6-f0c5-4b5a-b7d6-5cfc57c4e490 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Calling method '.version_select at 0x7f3e7b5b03a0>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:05.722356 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:05.742785 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:05.746942 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:05.750171 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e6fccaa6-6ecb-40ad-a7dc-ffdd8e1479f1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.468774 Jul 31 01:17:05.751974 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 427/856] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:05 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:05.816578 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-33c8c5b6-f0c5-4b5a-b7d6-5cfc57c4e490 tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c" status: 200 len: 352 microversion: 2.1 time: 0.105138 Jul 31 01:17:05.818422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 430/857] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:17:05 2022] GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c => generated 352 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:17:06.124910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:06.127121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:06.138027 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:06.138442 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:06.139830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:06.141122 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:06.417451 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:06.430877 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-19c7b4a0-8e57-4617-afe0-72624b070f34 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.312076 Jul 31 01:17:06.437555 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 428/858] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:06 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:06.789750 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:06.792436 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:06.812481 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:06.813513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:06.814908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:06.815792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:06.838715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-184aa485-a5ab-4775-bca5-01d33491945b tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Calling method '.version_select at 0x7f3e7b630e50>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:06.880923 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-184aa485-a5ab-4775-bca5-01d33491945b tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] HTTP exception thrown: Volume 042e58d7-86ec-4dac-a98d-75b71a23602c could not be found. Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-184aa485-a5ab-4775-bca5-01d33491945b tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Request method failure captured: Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c HTTP/1.1 Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dgP4S62iGKbwIRqUKfabcsMCuJ57T6mOue-_XGE1hUgLBJLTewiDIJgrDA7NTtKoWFo76fUzFqY0GKSsWYV20kyA8QKZL4Xxrzq5rsn7dpYLk4BQTWYiI71s9RCar1TBVx0EHpMdFOLNa81NFgWWO36l4s4oLc3V4iA-4mdLpocXfE Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: bb37e790a47c4e15a7e14fe6702ec1d5 Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesGetTestJSON-1024975970 Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:17:06.884848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/bb37e790a47c4e15a7e14fe6702ec1d5"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bb37e790a47c4e15a7e14fe6702ec1d5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bb37e790a47c4e15a7e14fe6702ec1d5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_bb37e790a47c4e15a7e14fe6702ec1d5", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesGetTestJSON-1024975970 Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: bb37e790a47c4e15a7e14fe6702ec1d5 Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesGetTestJSON-1024975970 Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesGetTestJSON-1024975970-project Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 952605aec2924e518736b1d11e1d9f3b Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesGetTestJSON-1024975970-project Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Volume 042e58d7-86ec-4dac-a98d-75b71a23602c could not be found. Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '042e58d7-86ec-4dac-a98d-75b71a23602c'} Jul 31 01:17:06.887724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 042e58d7-86ec-4dac-a98d-75b71a23602c could not be found. (HTTP 404) (Request-ID: req-0aedf89d-d1ac-47a0-ad42-f75395540ec8) Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:06.888759 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 31 01:17:06.889913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 042e58d7-86ec-4dac-a98d-75b71a23602c could not be found. Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 31 01:17:06.891638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 042e58d7-86ec-4dac-a98d-75b71a23602c could not be found. Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 042e58d7-86ec-4dac-a98d-75b71a23602c could not be found. Jul 31 01:17:06.893086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:17:06.894393 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-184aa485-a5ab-4775-bca5-01d33491945b tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] Returning 404 to user: Volume 042e58d7-86ec-4dac-a98d-75b71a23602c could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:17:06.895437 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-184aa485-a5ab-4775-bca5-01d33491945b tempest-VolumesGetTestJSON-1024975970 tempest-VolumesGetTestJSON-1024975970-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c" status: 404 len: 109 microversion: 2.1 time: 0.064842 Jul 31 01:17:06.896126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 429/859] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Jul 31 01:17:06 2022] GET /compute/v2.1/os-volumes/042e58d7-86ec-4dac-a98d-75b71a23602c => generated 109 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:17:07.073348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:07.082969 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:07.085315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:07.086910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-74b7a72c-6bf5-4b42-8a9d-84f140e5670f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.305281 Jul 31 01:17:07.087848 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 431/860] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:06 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:07.475842 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:07.480069 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:07.499873 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:07.501124 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:07.502870 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:07.504041 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:07.927462 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:07.986711 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1516f797-4a97-4131-bf3e-24288827dc1f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.516752 Jul 31 01:17:08.003446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 430/861] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:07 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:08.110852 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:08.111577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:08.151155 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:08.151660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:08.152520 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:08.152981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:08.424206 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:08.437854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:08.439494 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:08.440771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6cf44aee-0f29-483c-adf3-3c45739fabfa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1744 microversion: 2.20 time: 0.334393 Jul 31 01:17:08.452427 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 432/862] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:08 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1744 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:09.021486 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:09.023020 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:09.055805 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:09.056628 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:09.058274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:09.059007 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:09.470783 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:09.483480 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:09.484782 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:09.489867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-be137378-ad67-4eda-86aa-13132e45091c tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.481202 Jul 31 01:17:09.501389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:09.501857 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:09.502681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:09.503384 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:09.506504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 431/863] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:09 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:09.762918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:09.775297 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:09.781692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:09.783859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-41149b6e-91ab-4fb7-b97e-e66c4f46a116 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1744 microversion: 2.20 time: 0.304387 Jul 31 01:17:09.784976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 433/864] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:09 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1744 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:10.539496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:10.542088 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:10.566117 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:10.566919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:10.568412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:10.569485 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:10.821842 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:10.826802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:10.843198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:10.844613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:10.846330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:10.848085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:10.896751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:10.913968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:10.925678 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-911ac741-ebe4-4d86-ab59-921555452e28 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.392442 Jul 31 01:17:10.927492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 432/865] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:10 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:11.142200 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:11.175189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:11.182848 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:11.186558 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-70f11c9e-4f9c-4793-82e2-fcc6a5c85322 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1744 microversion: 2.20 time: 0.371726 Jul 31 01:17:11.203776 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 434/866] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:10 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1744 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:11.771092 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-3571ee76-c05a-447c-ae86-23c0406ec559 req-ef753620-e15a-41c4-bd4e-18cbf5142650 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cf3658-0cb6-48b8-b152-72b915385fbb", "name": "network-vif-plugged", "status": "completed", "tag": "38f41703-e2b2-4f28-9dc2-9118c346cfba"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:11.784955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-3571ee76-c05a-447c-ae86-23c0406ec559 req-ef753620-e15a-41c4-bd4e-18cbf5142650 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:11.785522 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-3571ee76-c05a-447c-ae86-23c0406ec559 req-ef753620-e15a-41c4-bd4e-18cbf5142650 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:11.814331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-3571ee76-c05a-447c-ae86-23c0406ec559 req-ef753620-e15a-41c4-bd4e-18cbf5142650 service nova] Creating event network-vif-plugged:38f41703-e2b2-4f28-9dc2-9118c346cfba for instance 69cf3658-0cb6-48b8-b152-72b915385fbb on hv-cinder-maste Jul 31 01:17:11.823874 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-3571ee76-c05a-447c-ae86-23c0406ec559 req-ef753620-e15a-41c4-bd4e-18cbf5142650 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055849 Jul 31 01:17:11.825012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 433/867] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:17:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:17:11.968444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:11.969799 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:11.999197 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:12.000336 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:12.002498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:12.003652 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:12.241098 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:12.244218 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:12.260511 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:12.261567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:12.263898 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:12.265089 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:12.412980 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:12.437441 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cc4e80f6-e772-426a-87f3-15151c698e69 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.476620 Jul 31 01:17:12.450051 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 435/868] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:11 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:12.686495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:12.705250 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:12.710892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:12.713789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d85bc24e-8c96-4583-94df-dbb76762e2b3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1744 microversion: 2.20 time: 0.482722 Jul 31 01:17:12.715107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 434/869] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:12 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1744 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:13.495245 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:13.504909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:13.524082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:13.525384 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:13.527316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:13.528485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:13.749796 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:13.750853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:13.780046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:13.782221 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:13.783708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:13.784483 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:14.107262 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:14.125771 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:14.129227 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:14.130829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-951a127e-8995-4e4c-ae90-e005b7ea846f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1744 microversion: 2.20 time: 0.391544 Jul 31 01:17:14.145860 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 435/870] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:13 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1744 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:14.157782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-d53b7c19-0036-4886-b62a-c92c8795a436 req-deaed650-107c-4972-b811-c208fbbbf095 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c03b1463-390f-430b-9bdf-64322c030504", "name": "network-vif-plugged", "status": "completed", "tag": "7d683e21-e88a-4b21-ba07-f00c5ff5031e"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:14.171967 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-d53b7c19-0036-4886-b62a-c92c8795a436 req-deaed650-107c-4972-b811-c208fbbbf095 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:14.173003 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-d53b7c19-0036-4886-b62a-c92c8795a436 req-deaed650-107c-4972-b811-c208fbbbf095 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:14.191329 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-d53b7c19-0036-4886-b62a-c92c8795a436 req-deaed650-107c-4972-b811-c208fbbbf095 service nova] Creating event network-vif-plugged:7d683e21-e88a-4b21-ba07-f00c5ff5031e for instance c03b1463-390f-430b-9bdf-64322c030504 on hv-cinder-maste Jul 31 01:17:14.200278 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-d53b7c19-0036-4886-b62a-c92c8795a436 req-deaed650-107c-4972-b811-c208fbbbf095 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051708 Jul 31 01:17:14.204144 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 436/871] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:17:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:17:14.288684 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:14.306354 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7a364aa8-4a80-4e96-b203-1df35604e53b tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1577 microversion: 2.1 time: 0.825273 Jul 31 01:17:14.310890 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 436/872] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:13 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1577 bytes in 832 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:15.165985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:15.167334 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:15.203235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:15.213387 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:15.215558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:15.216414 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:15.347341 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:15.348226 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:15.364262 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:15.364814 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:15.365753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:15.366462 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:15.793369 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:15.813370 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d518c747-8c36-439e-8c33-0e1561d9dc59 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1564 microversion: 2.1 time: 0.469315 Jul 31 01:17:15.815137 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 437/873] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:15 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1564 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:16.037360 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:16.058401 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:16.061310 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:16.063055 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-59be513a-54f0-4b13-b6e8-50f8a5924cef tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.906264 Jul 31 01:17:16.063997 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 437/874] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:15 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 909 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:17.123622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:17.132188 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:17.182529 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:17.200177 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:17.203573 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:17.204837 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:17.706226 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:17.737060 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:17.740337 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:17.757420 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0f483376-b3d1-4d9d-a49b-4b0b29c780cd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.665363 Jul 31 01:17:17.767958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 438/875] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:17 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:18.813179 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:18.815848 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:18.845721 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:18.846480 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:18.864040 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:18.867496 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:19.785494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:19.826535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:19.830958 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:19.848056 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-040fbe77-9e47-4c2d-8034-791c376e6e82 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 1.054372 Jul 31 01:17:19.849371 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 438/876] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:18 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 1062 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:20.894195 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:20.895493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:20.912731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:20.913654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:20.916432 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:20.917104 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:21.380489 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:21.405040 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:21.411072 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:21.416660 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-634c872d-49b1-4cc6-9c98-66ebab320248 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.526663 Jul 31 01:17:21.434803 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 439/877] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:20 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:22.446187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:22.446871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:22.461370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:22.472159 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:22.473064 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:22.473527 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:22.763933 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:22.766417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=100 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:22.781412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:22.783008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:22.788275 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0d825a39-b537-45da-b89d-23a05e8c3969 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.346396 Jul 31 01:17:22.805829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 439/878] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:22 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:23.849213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:23.853928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:23.872463 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:23.873236 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:23.874675 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:23.875545 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:24.344002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:24.437767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:24.441141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:24.459467 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1e974530-bd85-4fd6-a003-f4792197c90d tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.602922 Jul 31 01:17:24.471920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 440/879] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:23 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:24.738452 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d246f0a6-c93b-4d55-bcf8-5435c2b36a02 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Action: 'create', calling method: .version_select at 0x7f3e7b630dc0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1012408600", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB2VypN5+8rKjrU8KPOnZ3x/m6NcbkmLytZge8UKYWTkn2VdB+gkaW7Ad8nRz63yypRO1e3mAeYeAsGNnV8qtwMn3rdEraA6JFiMP5SMZaD4g+zWU6QH/ZX1yjfNt54u6A=="}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:24.775619 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-d246f0a6-c93b-4d55-bcf8-5435c2b36a02 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:17:24.781946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-d246f0a6-c93b-4d55-bcf8-5435c2b36a02 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for user 6a00c618dd284f159d88ad41c0390c5e and project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'key_pairs'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:17:24.834046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d246f0a6-c93b-4d55-bcf8-5435c2b36a02 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.484493 Jul 31 01:17:24.834844 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 440/880] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:17:24 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:17:25.506333 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:25.508010 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:25.514943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=33 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:25.519914 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:25.530999 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:25.534552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:25.535841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:25.883968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:25.909322 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:25.912675 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:25.915293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cc96d81c-9222-4d96-8a93-e6bace5014ff tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.413942 Jul 31 01:17:25.917027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 441/881] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:25 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:26.967549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:26.969721 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:26.978029 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=51,nova_api:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:27.005657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:27.007069 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:27.013613 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:27.014748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:27.379882 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:27.410703 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:27.413807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:27.421110 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8dc807bc-3285-44f4-97b4-6475390fa6df tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.462006 Jul 31 01:17:27.426772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 441/882] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:26 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:28.467930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:28.468779 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:28.481254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:28.485923 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:28.486975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:28.488027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:28.770970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:28.781302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:28.783451 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:28.784992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e796756d-28e1-4280-b9a1-9f1c667facf9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.322127 Jul 31 01:17:28.786189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 442/883] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:28 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:29.832684 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:29.833536 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:29.850170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:29.854869 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:29.856100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:29.856635 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:30.288098 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:30.312645 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:30.315160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:30.316980 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b837eb2d-21ed-446f-9009-2b07f2be995e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.489780 Jul 31 01:17:30.318749 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 442/884] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:29 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:31.346388 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:31.347006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:31.359273 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:31.359752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:31.360829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:31.361271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:31.698704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:31.722542 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:31.727313 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:31.730554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-612993d9-d800-4623-b2c7-e6a129404db9 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.390168 Jul 31 01:17:31.737015 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 443/885] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:31 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:32.778954 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:32.781593 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:32.796831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:32.798102 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:32.799504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:32.800578 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:33.037004 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:33.049711 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:33.056085 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:33.058426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-357a1ef7-3828-4f21-8f08-f071a1f429dd tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.289478 Jul 31 01:17:33.079458 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 443/886] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:32 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:34.087632 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:34.088892 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:34.096511 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:34.097308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:34.098560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:34.099198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:34.291732 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:34.301405 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:34.303227 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:34.304485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-37c97323-09ab-440f-af8f-facd2a022168 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1739 microversion: 2.20 time: 0.221230 Jul 31 01:17:34.305074 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 444/887] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:34 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1739 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:35.330887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:35.332153 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:35.335504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=4 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:35.343009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:35.343509 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:35.344420 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:35.344966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:35.563321 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:35.573987 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:35.575756 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:35.577008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-79946e87-cae0-4b32-afe3-8b7787b06802 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1721 microversion: 2.20 time: 0.252815 Jul 31 01:17:35.577598 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 444/888] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:35 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1721 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:36.602398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:36.603098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:36.612207 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:36.612661 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:36.613450 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:36.613893 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:36.804262 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:36.818644 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:36.821786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:36.824047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f91676f7-8846-4153-b399-d4b0e65e8552 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1721 microversion: 2.20 time: 0.225919 Jul 31 01:17:36.825380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 445/889] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:36 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1721 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:37.646061 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-20feb21d-0ec5-44fe-a260-62feffac645b req-3f9670be-1cb2-4ca7-99a1-ff7363d55247 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c03b1463-390f-430b-9bdf-64322c030504", "name": "network-vif-unplugged", "status": "completed", "tag": "7d683e21-e88a-4b21-ba07-f00c5ff5031e"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:37.664090 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-20feb21d-0ec5-44fe-a260-62feffac645b req-3f9670be-1cb2-4ca7-99a1-ff7363d55247 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:37.664539 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-20feb21d-0ec5-44fe-a260-62feffac645b req-3f9670be-1cb2-4ca7-99a1-ff7363d55247 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:37.680253 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-20feb21d-0ec5-44fe-a260-62feffac645b req-3f9670be-1cb2-4ca7-99a1-ff7363d55247 service nova] Creating event network-vif-unplugged:7d683e21-e88a-4b21-ba07-f00c5ff5031e for instance c03b1463-390f-430b-9bdf-64322c030504 on hv-cinder-maste Jul 31 01:17:37.686566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-20feb21d-0ec5-44fe-a260-62feffac645b req-3f9670be-1cb2-4ca7-99a1-ff7363d55247 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.049280 Jul 31 01:17:37.688036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 445/890] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:17:37 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:17:37.861375 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:37.862152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:37.870641 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:37.871343 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:37.873264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:37.873919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:38.063295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:38.073460 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:38.075434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:38.076759 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e929814e-4164-4809-90c5-802ea88500fa tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1724 microversion: 2.20 time: 0.219396 Jul 31 01:17:38.077389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 446/891] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:37 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1724 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:38.107645 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-325ef67f-0958-4a33-84bb-d0eef564df03 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "126477f4-e666-4bb9-b60c-ee29432e098e"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:38.111120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-325ef67f-0958-4a33-84bb-d0eef564df03 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:38.119028 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-325ef67f-0958-4a33-84bb-d0eef564df03 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:38.120310 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-325ef67f-0958-4a33-84bb-d0eef564df03 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:38.121202 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-325ef67f-0958-4a33-84bb-d0eef564df03 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:38.121755 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-325ef67f-0958-4a33-84bb-d0eef564df03 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:38.490084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-325ef67f-0958-4a33-84bb-d0eef564df03 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.388393 Jul 31 01:17:38.490951 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 446/892] 10.222.0.42 () {62 vars in 1364 bytes} [Sun Jul 31 01:17:38 2022] POST /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-volume_attachments => generated 188 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 31 01:17:38.643773 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-245f23ce-4823-4eaf-84d7-3712d703a6b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:38.644323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-245f23ce-4823-4eaf-84d7-3712d703a6b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:38.651621 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-245f23ce-4823-4eaf-84d7-3712d703a6b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:38.652125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-245f23ce-4823-4eaf-84d7-3712d703a6b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:38.652994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-245f23ce-4823-4eaf-84d7-3712d703a6b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:38.653446 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-245f23ce-4823-4eaf-84d7-3712d703a6b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:38.990522 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-245f23ce-4823-4eaf-84d7-3712d703a6b5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-volume_attachments/126477f4-e666-4bb9-b60c-ee29432e098e" status: 202 len: 0 microversion: 2.20 time: 0.350855 Jul 31 01:17:38.992616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 447/893] 10.222.0.42 () {60 vars in 1457 bytes} [Sun Jul 31 01:17:38 2022] DELETE /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-volume_attachments/126477f4-e666-4bb9-b60c-ee29432e098e => generated 0 bytes in 354 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:17:39.122704 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:39.123831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:39.136364 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:39.137120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:39.138543 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:39.139383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:39.183235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=117943) unshelve /opt/stack/nova/nova/compute/api.py:4531}} Jul 31 01:17:39.278338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR oslo.messaging._drivers.impl_rabbit [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [a2b70b2a-7550-46f9-bd7f-83506986fcad] AMQP server on 10.222.0.42:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 31 01:17:40.286692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO oslo.messaging._drivers.impl_rabbit [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [a2b70b2a-7550-46f9-bd7f-83506986fcad] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 39768. Jul 31 01:17:40.291896 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0f2272ab-a2a3-4256-b70d-a6b2d1885417 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "POST /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/action" status: 202 len: 0 microversion: 2.20 time: 1.177050 Jul 31 01:17:40.293868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 447/894] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Jul 31 01:17:39 2022] POST /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/action => generated 0 bytes in 1181 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:17:40.302838 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:40.304064 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:40.312455 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:40.313220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:40.314447 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:40.315013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:40.501731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:40.504571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:40.512366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:40.514279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:40.516074 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b11e0e64-5d64-48ea-a823-0d4318cfce38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1672 microversion: 2.20 time: 0.217964 Jul 31 01:17:40.517179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 448/895] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:40 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1672 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:41.556036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:41.557641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:41.569856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:41.570352 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:41.571232 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:41.571887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:41.819012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:41.828928 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:41.831060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:41.832042 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-68ac18c7-98e6-4804-8e9d-b022c07e54f2 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1674 microversion: 2.20 time: 0.283857 Jul 31 01:17:41.832624 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 448/896] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:41 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1674 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:42.870343 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:42.871096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:42.880109 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:42.880854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:42.881805 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:42.882497 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:43.145264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:43.164422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:43.167000 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:43.169254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1cd6e9dc-c033-4ace-a7ed-232e8bd43768 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.304015 Jul 31 01:17:43.171205 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 449/897] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:42 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:44.216228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:44.218292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:44.229155 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:44.229708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:44.230698 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:44.235603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:44.535036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:44.545869 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:44.549563 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:44.553955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4a20d716-2dbf-4ad8-81b6-9b4a6e19e197 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.344765 Jul 31 01:17:44.557917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 449/898] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:44 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:45.603661 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:45.605781 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:45.621003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:45.622665 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:45.625816 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:45.626622 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:45.847259 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:45.862208 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:45.865285 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:45.867492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a6886cb0-ed9d-4262-8a71-5c436d755ce8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.270986 Jul 31 01:17:45.868504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 450/899] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:45 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:46.871263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1154469419", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "networks": [{"uuid": "5461ee14-86f9-434e-a42e-9a54983e738c"}]}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:46.912488 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:46.914173 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:46.934571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:46.935252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:46.936368 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:46.937125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:46.959960 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] validate_networks() for [('5461ee14-86f9-434e-a42e-9a54983e738c', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:17:47.170271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:47.184924 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:47.186716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:47.188124 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c039564a-4a95-4008-ad97-3d1d70ffcb10 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.283580 Jul 31 01:17:47.203397 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 451/900] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:46 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:47.593401 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:17:47.594550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:17:47.595334 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:17:47.596152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:17:47.596956 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:17:47.614378 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:47.615046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:47.616605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:47.617398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:47.652716 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Getting quotas for project 0f2417d70b00404f8e37d52f694f9a41. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:17:47.661100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Getting quotas for user 943af9a17a554646808ac2f2598a9adf and project 0f2417d70b00404f8e37d52f694f9a41. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:17:47.686263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:17:47.691082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:17:47.786325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-60d6cab0-ecf4-4523-81ad-cb7faafff908 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.923679 Jul 31 01:17:47.790226 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 450/901] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:17:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 929 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:17:47.810964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-913f37ae-72b6-4a69-b0ba-e6554a15434f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:47.812201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-913f37ae-72b6-4a69-b0ba-e6554a15434f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:47.962356 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-913f37ae-72b6-4a69-b0ba-e6554a15434f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:47.983063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-913f37ae-72b6-4a69-b0ba-e6554a15434f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1219 microversion: 2.1 time: 0.181241 Jul 31 01:17:47.984323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 452/902] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:47 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1219 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:48.219001 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:48.221123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:48.233337 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:48.234528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:48.236327 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:48.237537 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:48.525812 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:48.544834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:48.548725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:48.551238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0272591f-06a4-46ec-9103-9aeb37dbc70f tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.341809 Jul 31 01:17:48.553765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 451/903] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:48 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:49.014920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:49.015726 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:49.024076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:49.024547 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:49.025436 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:49.025914 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:49.110281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:49.118933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8b8ba926-5229-427e-b3c0-2c652a792fea tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.108391 Jul 31 01:17:49.120513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 453/904] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:49 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:49.594985 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:49.595736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:49.602957 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:49.603672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:49.605239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:49.605681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:49.851229 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:49.864503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:49.866276 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:49.867545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e8c65d46-c530-4195-81c7-9294b2587dd1 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.276825 Jul 31 01:17:49.868146 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 452/905] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:49 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:50.157097 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:50.158469 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:50.170307 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:50.171136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:50.172604 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:50.173415 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:50.290804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:50.301752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-047ea878-8501-410d-9964-baed21285d7e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.152647 Jul 31 01:17:50.302643 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 454/906] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:50 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:50.901857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:50.903079 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:50.916186 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:50.917732 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:50.919182 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:50.919844 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:51.195671 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:51.211638 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:51.213772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:51.215148 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-267b1497-9366-4ac0-9716-13faf09c7f71 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.322063 Jul 31 01:17:51.219408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 453/907] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:50 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:51.340919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:51.344025 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:51.356640 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:51.357995 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:51.359891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:51.361025 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:51.413635 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1575133896", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1012408600", "security_groups": [{"name": "tempest-secgroup-smoke-108034258"}], "block_device_mapping_v2": [{"uuid": "7e83a856-9bae-46b8-a054-f0db55b2f035", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3baf6cd9-bfb5-4744-8ae2-8508343cba8a"}]}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:17:51.463876 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:51.473893 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b1c12fbf-10da-41a1-b85a-ef0dbff70cc9 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.139083 Jul 31 01:17:51.475012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 455/908] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:51 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:51.636863 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] validate_networks() for [('3baf6cd9-bfb5-4744-8ae2-8508343cba8a', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:17:52.170758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:17:52.171658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:17:52.171658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:17:52.172068 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:17:52.172469 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:17:52.184204 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:52.184673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:52.185458 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:52.185887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:52.221336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:17:52.227054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for user 6a00c618dd284f159d88ad41c0390c5e and project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:17:52.245171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:52.248525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:52.262892 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:52.263766 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:52.265264 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:52.265997 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:52.364246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:17:52.367243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='7e83a856-9bae-46b8-a054-f0db55b2f035',volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:17:52.582077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:52.601193 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:52.605985 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:52.607923 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-57098487-1bae-4547-8d67-14f60e07fc51 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.370057 Jul 31 01:17:52.608698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 456/909] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:52 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:52.635254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:52.638617 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:52.667339 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:52.669098 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:52.671131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:52.672116 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:52.680681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0659f490-9536-4f65-b259-f6434e0dc65e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.287035 Jul 31 01:17:52.692150 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 454/910] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:17:51 2022] POST /compute/v2.1/servers => generated 405 bytes in 1300 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:17:52.700846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-51724aef-0996-4187-acfb-12e5002d19e3 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:52.702054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-51724aef-0996-4187-acfb-12e5002d19e3 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:52.816589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-51724aef-0996-4187-acfb-12e5002d19e3 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:52.834223 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-51724aef-0996-4187-acfb-12e5002d19e3 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1095 microversion: 2.1 time: 0.140153 Jul 31 01:17:52.835768 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 455/911] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:52 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1095 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:52.897263 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:52.917657 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-06c214f8-80a8-43c6-9b33-4c236df6a3fd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.306687 Jul 31 01:17:52.931397 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 457/912] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:52 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:53.654044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:53.654894 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:53.664363 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:53.665245 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:53.667050 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:53.667925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:53.672163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=87,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:53.860129 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:53.872602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:53.873813 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:53.876885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:53.884542 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:53.886505 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:53.886830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-452323d2-3690-4b23-ad76-5968b8b714d0 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.236485 Jul 31 01:17:53.887546 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:53.887962 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 456/913] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:53 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:53.889155 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:53.889790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:53.960152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:53.961149 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:53.969143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:53.970050 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:53.971235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:53.971979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:54.009468 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:54.027188 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dbc4982f-2b2d-4409-a697-dd9d060e1735 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1145 microversion: 2.1 time: 0.162114 Jul 31 01:17:54.029574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 458/914] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:53 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1145 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:54.055510 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:54.066789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-17ac9c05-dd9f-4822-9754-42c217db4c46 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.111469 Jul 31 01:17:54.068026 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 457/915] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:53 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:54.912913 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:54.913536 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:54.923032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:54.923806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:54.925176 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:54.925911 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:55.061867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:55.062940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:55.071074 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:55.071431 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:55.072274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:55.073426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:55.100467 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:55.114888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:55.117857 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:55.120483 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b0b76dd1-3eaf-4eac-b644-6a0c89543ba8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.210825 Jul 31 01:17:55.121610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 459/916] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:54 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:55.128554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:55.129389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:55.139416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:55.139416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:55.139416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:55.140158 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:55.162779 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:55.173494 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-80fe2743-640c-4a7f-891c-d2e7129fe7f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1145 microversion: 2.1 time: 0.116624 Jul 31 01:17:55.173989 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 458/917] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:55 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1145 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:55.221658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:55.230613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d6047b43-d450-4143-b92c-6981e8929491 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.107602 Jul 31 01:17:55.231222 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 460/918] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:55 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:56.160944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:56.161735 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:56.169247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:56.169733 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:56.170573 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:56.171102 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:56.198700 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:56.200154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:56.202440 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=42 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:56.208498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:56.208947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:56.209743 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:56.210187 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:56.292135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:56.306976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8e75ccb8-9f91-4c23-95b7-88c75057fb7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1145 microversion: 2.1 time: 0.114316 Jul 31 01:17:56.308716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 461/919] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:56 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1145 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:56.316974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:56.318146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:56.331034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:56.332016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:56.333803 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:56.334640 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:56.367775 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:56.377826 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:56.379612 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:56.380865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9a34f53d-bb77-4b21-beaa-59b0ee6f1f38 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.224371 Jul 31 01:17:56.381462 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 459/920] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:56 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:56.429608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:56.438918 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7f0a6d9f-336e-4b81-8785-7bf00c63fe7d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.128698 Jul 31 01:17:56.439562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 462/921] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:56 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:57.330112 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:57.330762 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:57.333306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=68,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:57.340066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:57.340517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:57.341331 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:57.341779 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:57.413077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:57.422259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6e9a6ac2-0b8a-4e9b-98b2-8750f1d372c2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1145 microversion: 2.1 time: 0.096457 Jul 31 01:17:57.422557 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:57.423296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 460/922] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:57 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1145 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:57.423536 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:57.437271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:57.438463 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:57.439881 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:57.440632 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:57.463259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:57.466714 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:57.476343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:57.476867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:57.477939 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:57.478566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:57.567235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:57.576396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8bbff895-156f-4ca5-994a-13999ebe5fd1 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.117146 Jul 31 01:17:57.577117 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 461/923] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:57 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:57.654753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:57.664072 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:57.665753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:57.667111 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1844b922-d6db-4ae8-bc3c-cb65f8fbc20b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.251921 Jul 31 01:17:57.667759 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 463/924] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:57 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:17:58.446220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:58.446921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:58.455705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:58.456163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:58.456967 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:58.457409 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:58.532423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:58.540784 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-45e603a6-050b-4b6c-84ef-39cbd4eb5f37 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1145 microversion: 2.1 time: 0.099494 Jul 31 01:17:58.541582 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 462/925] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:58 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1145 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:58.599762 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:58.600392 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:58.602414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:58.609710 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:58.610363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:58.611277 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:58.611750 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:58.689416 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:58.691198 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:58.692024 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:58.698490 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-743ed2a6-c984-4cff-a6ef-40410ffebabd tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.102643 Jul 31 01:17:58.698756 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:58.699064 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:58.699287 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 464/926] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:58 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:17:58.699900 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:58.700343 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:58.863736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:58.874057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:58.875695 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:58.876902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-9f8f4fad-a505-40b8-978a-12f272aeb471 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.189689 Jul 31 01:17:58.877487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 463/927] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:58 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:17:59.212984 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1757eea0-c513-41c4-8745-15eef7e74c16 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:59.215234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1757eea0-c513-41c4-8745-15eef7e74c16 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:59.237696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1757eea0-c513-41c4-8745-15eef7e74c16 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.238498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1757eea0-c513-41c4-8745-15eef7e74c16 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:59.239900 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1757eea0-c513-41c4-8745-15eef7e74c16 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.240706 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1757eea0-c513-41c4-8745-15eef7e74c16 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:59.289616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1757eea0-c513-41c4-8745-15eef7e74c16 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/ips" status: 200 len: 150 microversion: 2.1 time: 0.085814 Jul 31 01:17:59.300734 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 465/928] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:17:59 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/ips => generated 150 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:17:59.578137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:59.579395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:59.592015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.592015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:59.593048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.593506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:59.698626 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:59.708774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-60a75956-0d6d-4119-a736-7d0113e8d5f0 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1145 microversion: 2.1 time: 0.137459 Jul 31 01:17:59.709374 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 464/929] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:59 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1145 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:59.732372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:59.733618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:59.742326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.742999 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:59.743994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.745380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:59.818661 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:17:59.821094 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=100 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:17:59.832216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d4deafff-1e55-4af0-8630-079a0ea08ac8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1223 microversion: 2.1 time: 0.105197 Jul 31 01:17:59.833141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 466/930] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:17:59 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1223 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:17:59.900508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:17:59.901158 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:17:59.907795 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.908254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:17:59.909044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:17:59.909480 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:00.059947 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:00.069581 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:00.071172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:00.072587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ed48b030-1fe9-4f3a-8e72-83324cf5c134 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.176148 Jul 31 01:18:00.073305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 465/931] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:17:59 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:18:00.768493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:00.771485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:00.778697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ac84f2f6-797b-4c60-a117-c5f1ce4a6b93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ab804913-ba0c-4e4f-850d-48e1b96ac5af"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:00.779692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:00.779692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:00.780053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:00.780635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:00.796818 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ac84f2f6-797b-4c60-a117-c5f1ce4a6b93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:00.822039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac84f2f6-797b-4c60-a117-c5f1ce4a6b93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:00.829909 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac84f2f6-797b-4c60-a117-c5f1ce4a6b93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:00.833518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac84f2f6-797b-4c60-a117-c5f1ce4a6b93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:00.834884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ac84f2f6-797b-4c60-a117-c5f1ce4a6b93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:00.850042 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:00.860088 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b2d9c534-4118-4327-8765-cd31e915385a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1137 microversion: 2.1 time: 0.124043 Jul 31 01:18:00.861731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 467/932] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:00 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1137 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:00.868260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:00.869168 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:00.877175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:00.878095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:00.879173 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:00.879864 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:00.996162 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:01.005164 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7b6bfbdb-bcec-45d9-9da6-e89b2ef5641f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1215 microversion: 2.1 time: 0.141903 Jul 31 01:18:01.006230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 468/933] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:00 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1215 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:01.097326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:01.097861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:01.104873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:01.105331 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:01.106164 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:01.106609 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:01.258216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:01.267506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:01.269063 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:01.270308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b20a7ef5-3326-417e-8093-adfa4393288e tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.177186 Jul 31 01:18:01.270895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 469/934] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:18:01 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:18:01.857596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ac84f2f6-797b-4c60-a117-c5f1ce4a6b93 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "POST /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.085712 Jul 31 01:18:01.859850 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 466/935] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:18:00 2022] POST /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments => generated 194 bytes in 1089 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:01.909468 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:01.914530 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:01.953136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:01.957827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:01.960028 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:01.965344 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:02.045566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:02.046837 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:02.060856 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:02.061970 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:02.069389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:02.071917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:02.201802 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:02.204359 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:02.210110 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ca32c062-7277-4ec9-8121-2fed6a9118c6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1201 microversion: 2.1 time: 0.312622 Jul 31 01:18:02.216194 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=4 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:02.223965 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 470/936] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:01 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1201 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:02.249313 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-92d80f0f-53e0-438d-88e2-8b6d65e7c50a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1277 microversion: 2.1 time: 0.209603 Jul 31 01:18:02.263392 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 467/937] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:02 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1277 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:02.303937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:02.305131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:02.317983 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:02.319216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:02.320915 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:02.321881 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:02.845390 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:02.854642 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:02.856381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:02.857610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ae7ffc66-0b70-4a4c-81ff-b2dc265e0e0c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.559011 Jul 31 01:18:02.858430 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 471/938] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:18:02 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:18:03.236875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:03.237836 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:03.242050 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=14 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:03.251957 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:03.252599 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:03.253696 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:03.254760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:03.280140 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:03.281054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:03.292333 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:03.293229 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:03.294144 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:03.294653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:03.359071 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:03.376336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-16d692b8-0a18-4f43-8157-c0a06627e119 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1211 microversion: 2.1 time: 0.146164 Jul 31 01:18:03.377506 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 468/939] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:03 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1211 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:03.403348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:03.415906 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cb166223-69b6-4d5c-9068-1b393e25ed07 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1277 microversion: 2.1 time: 0.142601 Jul 31 01:18:03.417774 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 472/940] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:03 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1277 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:03.851691 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-3a31a89a-ffb4-44ab-8bec-44c7b6b7c75f req-f1067cf8-8c09-459d-9abe-8dbba551a16f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c03b1463-390f-430b-9bdf-64322c030504", "name": "network-vif-plugged", "status": "completed", "tag": "7d683e21-e88a-4b21-ba07-f00c5ff5031e"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:03.886230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:03.889153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:03.907862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:03.910811 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-3a31a89a-ffb4-44ab-8bec-44c7b6b7c75f req-f1067cf8-8c09-459d-9abe-8dbba551a16f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:03.911980 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:03.912302 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-3a31a89a-ffb4-44ab-8bec-44c7b6b7c75f req-f1067cf8-8c09-459d-9abe-8dbba551a16f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:03.914458 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:03.915331 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:03.936257 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-3a31a89a-ffb4-44ab-8bec-44c7b6b7c75f req-f1067cf8-8c09-459d-9abe-8dbba551a16f service nova] Creating event network-vif-plugged:7d683e21-e88a-4b21-ba07-f00c5ff5031e for instance c03b1463-390f-430b-9bdf-64322c030504 on hv-cinder-maste Jul 31 01:18:03.949215 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-3a31a89a-ffb4-44ab-8bec-44c7b6b7c75f req-f1067cf8-8c09-459d-9abe-8dbba551a16f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103729 Jul 31 01:18:03.950978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 469/941] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:04.413012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:04.414192 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:04.422348 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:04.422930 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:04.423889 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:04.424395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:04.492674 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:04.513243 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:04.516225 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:04.517827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-258b1a57-085a-46a2-b339-1e26b0d18ca6 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.639615 Jul 31 01:18:04.518712 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 473/942] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:18:03 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:18:04.528109 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:04.529753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:04.540716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:04.542000 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:04.543861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:04.545154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:04.843567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:04.852836 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f9ca3762-51ff-473b-891f-cefe63cc9637 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.444649 Jul 31 01:18:04.853707 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 470/943] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:04 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:05.016336 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:05.029572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-30b4e14a-3ba7-452e-88c7-b7b5b7ca1871 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.509093 Jul 31 01:18:05.030592 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 474/944] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:04 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:05.356389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-35e550d0-dd9e-4207-a6f8-4ad852eec52b req-d67c9236-2907-472f-a9ff-db9acc6de6b6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3978683e-cb02-4c81-b2b9-10544bb204a9", "tag": "3c5adaa8-afed-461f-9a44-0d7402226cd4"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:05.381426 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-35e550d0-dd9e-4207-a6f8-4ad852eec52b req-d67c9236-2907-472f-a9ff-db9acc6de6b6 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:05.382252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-35e550d0-dd9e-4207-a6f8-4ad852eec52b req-d67c9236-2907-472f-a9ff-db9acc6de6b6 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:05.405674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-35e550d0-dd9e-4207-a6f8-4ad852eec52b req-d67c9236-2907-472f-a9ff-db9acc6de6b6 service nova] Creating event network-changed:3c5adaa8-afed-461f-9a44-0d7402226cd4 for instance 3978683e-cb02-4c81-b2b9-10544bb204a9 on hv-cinder-maste Jul 31 01:18:05.414400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-35e550d0-dd9e-4207-a6f8-4ad852eec52b req-d67c9236-2907-472f-a9ff-db9acc6de6b6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072219 Jul 31 01:18:05.415518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 471/945] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:05.570131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:05.570790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:05.583090 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:05.583928 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:05.585370 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:05.586192 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:05.885750 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:05.887867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:05.904311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:05.905087 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:05.906779 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:05.907853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:06.029888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:06.061731 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:06.074687 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:06.078330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1170cfbd-38ef-46d9-8165-689e1556119c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1730 microversion: 2.20 time: 0.511457 Jul 31 01:18:06.097257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 475/946] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:18:05 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1730 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:18:06.118040 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:06.119910 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:06.132751 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:06.133843 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:06.135540 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:06.136563 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:06.245683 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:06.249408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:06.267243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-137d9cf8-89b8-494c-9dad-e93dbcb8c054 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.391374 Jul 31 01:18:06.280414 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 472/947] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:05 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:06.495559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:06.510239 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dc74428b-d989-4b41-9653-17d965fb30cf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.406893 Jul 31 01:18:06.511844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 476/948] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:06 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:07.111696 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:07.112456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:07.121333 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:07.121835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:07.122744 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:07.123222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:07.298539 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:07.299487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:07.307282 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:07.307861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:07.308797 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:07.309367 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:07.503075 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:07.515095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:07.518662 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:07.520452 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-814a7286-f299-4684-a3cb-d645cd12fa15 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1717 microversion: 2.20 time: 0.413085 Jul 31 01:18:07.521039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 473/949] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:18:07 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1717 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:18:07.526453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-ea08c8a8-9c98-4d97-96b2-341ce6847afc req-3de1843f-29a9-4595-a6e2-53bcf9acba64 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d3c5a74-7a04-4f8b-a068-e68566b7d494", "tag": "55257163-7d49-4d57-9f0c-9a47106164ab"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:07.537297 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-ea08c8a8-9c98-4d97-96b2-341ce6847afc req-3de1843f-29a9-4595-a6e2-53bcf9acba64 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:07.537740 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-ea08c8a8-9c98-4d97-96b2-341ce6847afc req-3de1843f-29a9-4595-a6e2-53bcf9acba64 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:07.551851 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-ea08c8a8-9c98-4d97-96b2-341ce6847afc req-3de1843f-29a9-4595-a6e2-53bcf9acba64 service nova] Creating event network-changed:55257163-7d49-4d57-9f0c-9a47106164ab for instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 on hv-cinder-maste Jul 31 01:18:07.559020 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-ea08c8a8-9c98-4d97-96b2-341ce6847afc req-3de1843f-29a9-4595-a6e2-53bcf9acba64 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035934 Jul 31 01:18:07.560268 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 474/950] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:07.566806 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:07.567541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:07.578408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:07.579172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:07.580922 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:07.581617 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:07.615827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:07.625411 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2a2114bd-c74d-4de2-aee5-4a4337de2833 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.332388 Jul 31 01:18:07.625987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 477/951] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:07 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:07.767337 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:07.775491 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b91aabc6-0218-4136-b8b6-fe6005e3d988 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.213256 Jul 31 01:18:07.776119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 475/952] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:07 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:08.656693 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:08.658081 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:08.670602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:08.671438 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:08.672937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:08.673752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:08.799105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:08.800092 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:08.810960 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:08.811179 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:08.812758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:08.813251 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:08.896919 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:08.912006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a99d5bb3-da5f-4cda-8627-c5d45c66cfc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.261958 Jul 31 01:18:08.913103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 478/953] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:08 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:09.021621 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:09.036764 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-76c7a79a-c253-47b6-9c7c-4812b373bdcf tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.242309 Jul 31 01:18:09.037817 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 476/954] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:08 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:09.940358 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:09.941188 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:09.950600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:09.951695 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:09.952922 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:09.953573 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:10.075413 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:10.076921 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:10.087150 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:10.087807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:10.088838 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:10.089361 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:10.110691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:10.120792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-edded4f5-7d61-453b-983e-abd5c5797bb1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.185196 Jul 31 01:18:10.121506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 479/955] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:09 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:10.257629 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:10.267108 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-72455714-6818-4a61-a5d3-ec517b5fc804 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.196805 Jul 31 01:18:10.268152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 477/956] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:10 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:11.144258 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:11.144841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:11.152962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:11.153466 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:11.154487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:11.154970 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:11.293336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:11.294541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:11.306566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:11.308104 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:11.309203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:11.309938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:11.518541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:11.533338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a4cfb38a-8e55-4cf8-a267-1416e3fffa96 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.246182 Jul 31 01:18:11.534357 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 478/957] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:11 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:11.856117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:11.876544 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dc610069-00c7-418c-8e0c-5ccc6502f166 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.735907 Jul 31 01:18:11.883388 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 480/958] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:11 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 745 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:11.930139 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-369fa719-83ee-4350-b6a2-c20ff8862f67 req-44b36f13-d46a-4e0a-bf62-1e347ca15b56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3978683e-cb02-4c81-b2b9-10544bb204a9", "name": "network-vif-plugged", "status": "completed", "tag": "3c5adaa8-afed-461f-9a44-0d7402226cd4"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:11.952279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-369fa719-83ee-4350-b6a2-c20ff8862f67 req-44b36f13-d46a-4e0a-bf62-1e347ca15b56 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:11.953037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-369fa719-83ee-4350-b6a2-c20ff8862f67 req-44b36f13-d46a-4e0a-bf62-1e347ca15b56 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:11.991030 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-369fa719-83ee-4350-b6a2-c20ff8862f67 req-44b36f13-d46a-4e0a-bf62-1e347ca15b56 service nova] Creating event network-vif-plugged:3c5adaa8-afed-461f-9a44-0d7402226cd4 for instance 3978683e-cb02-4c81-b2b9-10544bb204a9 on hv-cinder-maste Jul 31 01:18:12.008035 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-369fa719-83ee-4350-b6a2-c20ff8862f67 req-44b36f13-d46a-4e0a-bf62-1e347ca15b56 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085693 Jul 31 01:18:12.009435 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 479/959] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:12.256908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:12.257770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:12.273253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.273829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:12.274745 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.275278 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:12.405134 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lazy-loading 'services' on Instance uuid 29c5be43-d161-49e2-8214-6fff60151b8b {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:18:12.529926 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fc81e3b2-5dd2-4209-8c9c-82015c0e98f7 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments/ab804913-ba0c-4e4f-850d-48e1b96ac5af" status: 202 len: 0 microversion: 2.1 time: 0.277570 Jul 31 01:18:12.530668 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 481/960] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:18:12 2022] DELETE /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments/ab804913-ba0c-4e4f-850d-48e1b96ac5af => generated 0 bytes in 279 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 31 01:18:12.540746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b6b1e06b-2de3-4adb-82b4-0c0cb4a976f3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:12.543545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b6b1e06b-2de3-4adb-82b4-0c0cb4a976f3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:12.555891 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6b1e06b-2de3-4adb-82b4-0c0cb4a976f3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.556945 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6b1e06b-2de3-4adb-82b4-0c0cb4a976f3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:12.558844 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6b1e06b-2de3-4adb-82b4-0c0cb4a976f3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.559082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b6b1e06b-2de3-4adb-82b4-0c0cb4a976f3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:12.567257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:12.568155 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:12.576578 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.577156 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:12.578120 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.578891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:12.611949 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b6b1e06b-2de3-4adb-82b4-0c0cb4a976f3 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075443 Jul 31 01:18:12.613017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 480/961] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:18:12 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:18:12.752492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:12.755255 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:12.763054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e6a0b021-a5ff-4094-8b1b-7ee16e7aa576 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.200758 Jul 31 01:18:12.767581 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 482/962] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:12 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:12.913637 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:12.914790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:12.926323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.926834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:12.927723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:12.928287 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:13.109852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:13.118629 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-020fc055-d0c7-42cc-a438-bef0d349ecab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.212731 Jul 31 01:18:13.119305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 481/963] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:12 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:13.631182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-548e63ce-6ecc-4bdf-8fed-98c38b0a0b36 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:13.633572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-548e63ce-6ecc-4bdf-8fed-98c38b0a0b36 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:13.644498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-548e63ce-6ecc-4bdf-8fed-98c38b0a0b36 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:13.645293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-548e63ce-6ecc-4bdf-8fed-98c38b0a0b36 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:13.646190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-548e63ce-6ecc-4bdf-8fed-98c38b0a0b36 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:13.646677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-548e63ce-6ecc-4bdf-8fed-98c38b0a0b36 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:13.685722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-548e63ce-6ecc-4bdf-8fed-98c38b0a0b36 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058748 Jul 31 01:18:13.686373 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 483/964] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:18:13 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:13.785443 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:13.786487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:13.799144 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:13.800017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:13.801388 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:13.802213 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:14.025621 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:14.035526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b336ea0c-e314-4a15-8376-ee612fd08e0b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1319 microversion: 2.1 time: 0.254828 Jul 31 01:18:14.036103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 482/965] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:13 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1319 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:14.142020 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:14.142801 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:14.150638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:14.151878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:14.152827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:14.153309 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:14.321132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:14.338018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3528c5b2-972e-4b65-a697-900227ae32ad tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.200072 Jul 31 01:18:14.339172 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 484/966] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:14 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:14.701901 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-aeb5213e-c6ba-4937-bb9a-9da83c234d70 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:14.704409 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-aeb5213e-c6ba-4937-bb9a-9da83c234d70 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:14.713024 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeb5213e-c6ba-4937-bb9a-9da83c234d70 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:14.713608 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeb5213e-c6ba-4937-bb9a-9da83c234d70 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:14.714723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeb5213e-c6ba-4937-bb9a-9da83c234d70 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:14.715403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aeb5213e-c6ba-4937-bb9a-9da83c234d70 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:14.753884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-aeb5213e-c6ba-4937-bb9a-9da83c234d70 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056388 Jul 31 01:18:14.754485 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 483/967] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:18:14 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:18:15.061157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:15.062311 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:15.071897 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.072664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:15.073827 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.074436 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:15.253503 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:15.263294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0ccf9c39-47ac-4182-ac78-c28c642dd73a tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1507 microversion: 2.1 time: 0.208113 Jul 31 01:18:15.264070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 485/968] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:15 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1507 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:15.364698 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:15.366017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:15.377572 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.378365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:15.380095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.380869 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:15.543240 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:15.557992 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e6e1b1dc-5bd1-4ab3-8de6-604ccf601e47 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.200489 Jul 31 01:18:15.559093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 484/969] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:15 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:15.770663 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d6411e8c-59a7-481e-b89c-29c59e04377d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:15.773958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d6411e8c-59a7-481e-b89c-29c59e04377d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:15.796391 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6411e8c-59a7-481e-b89c-29c59e04377d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.797482 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6411e8c-59a7-481e-b89c-29c59e04377d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:15.799070 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6411e8c-59a7-481e-b89c-29c59e04377d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.799950 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d6411e8c-59a7-481e-b89c-29c59e04377d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:15.855639 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d6411e8c-59a7-481e-b89c-29c59e04377d tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.087523 Jul 31 01:18:15.857013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 486/970] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:18:15 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b/os-volume_attachments => generated 25 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 31 01:18:15.961515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:15.962279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:15.969824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.970346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:15.971307 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:15.971865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:16.001268 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:18:16.097666 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-04e59aea-a4eb-4ba7-ba71-208c63a0f883 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 204 len: 0 microversion: 2.1 time: 0.140153 Jul 31 01:18:16.098541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 485/971] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:18:15 2022] DELETE /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:18:16.110335 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:16.111171 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:16.118075 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:16.118527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:16.119331 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:16.120406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:16.291410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:16.300855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-18af522d-8275-454b-86fe-d0cff5c1c829 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1682 microversion: 2.1 time: 0.194965 Jul 31 01:18:16.301478 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 487/972] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:16 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1682 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:16.584063 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:16.584778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:16.594336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:16.594785 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:16.595630 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:16.596066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:16.780570 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:16.789764 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ed57a3f4-baa7-4a0c-b025-1120c9bc0c75 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.210132 Jul 31 01:18:16.790348 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 486/973] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:16 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:17.334800 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:17.336367 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:17.345182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:17.345743 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:17.346677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:17.347199 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:17.583202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:17.594682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-29b62f36-2ad1-4deb-8014-75dd743ab3f4 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1682 microversion: 2.1 time: 0.263666 Jul 31 01:18:17.601822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 488/974] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:17 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1682 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:17.810673 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:17.811729 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:17.820321 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:17.820928 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:17.821853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:17.822430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:18.006438 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:18.022072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2e570cd6-9549-49db-afe9-b10a0862d8d7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.214792 Jul 31 01:18:18.023247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 487/975] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:17 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:18.616301 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:18.617007 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:18.625342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:18.625819 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:18.626715 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:18.627168 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:18.792403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:18.802166 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-424cbbbc-d103-4054-8003-5178696e341b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1682 microversion: 2.1 time: 0.189853 Jul 31 01:18:18.803042 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 489/976] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:18 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1682 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:19.046819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:19.047860 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:19.055812 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:19.056421 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:19.057459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:19.058080 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:19.231546 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:19.245898 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5d2fd3d5-f2bc-4c28-81e2-d19e72142b72 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.203105 Jul 31 01:18:19.246870 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 488/977] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:19 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:19.841457 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:19.843625 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:19.858506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:19.860055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:19.862035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:19.862709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:20.192485 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:20.203273 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7308e3da-ed98-46fe-af64-576a14c74049 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1682 microversion: 2.1 time: 0.369539 Jul 31 01:18:20.204137 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 490/978] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:19 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1682 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:20.285219 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:20.286579 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:20.299932 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:20.300885 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:20.302441 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:20.303229 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:20.312076 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:20.527094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:20.542656 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-481f7b53-5b7a-44a0-9236-d8cb33086a01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.265716 Jul 31 01:18:20.543637 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 489/979] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:20 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:20.857347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-f1e828c3-8cd9-4ae2-b331-d497111fe3e7 req-9bca8332-107c-4ae9-9dfe-ba514ed89f11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29c5be43-d161-49e2-8214-6fff60151b8b", "name": "network-vif-deleted", "tag": "658533d1-a639-468a-a657-a8f9bc1703b3"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:20.879271 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f1e828c3-8cd9-4ae2-b331-d497111fe3e7 req-9bca8332-107c-4ae9-9dfe-ba514ed89f11 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:20.880539 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f1e828c3-8cd9-4ae2-b331-d497111fe3e7 req-9bca8332-107c-4ae9-9dfe-ba514ed89f11 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:20.904009 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-f1e828c3-8cd9-4ae2-b331-d497111fe3e7 req-9bca8332-107c-4ae9-9dfe-ba514ed89f11 service nova] Creating event network-vif-deleted:658533d1-a639-468a-a657-a8f9bc1703b3 for instance 29c5be43-d161-49e2-8214-6fff60151b8b on hv-cinder-maste Jul 31 01:18:20.910713 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-f1e828c3-8cd9-4ae2-b331-d497111fe3e7 req-9bca8332-107c-4ae9-9dfe-ba514ed89f11 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061169 Jul 31 01:18:20.912329 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 491/980] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:21.227070 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:21.227854 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:21.237208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:21.237765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:21.238639 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:21.239286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:21.281298 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:21.281773 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:21.282296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lazy-loading 'fault' on Instance uuid 29c5be43-d161-49e2-8214-6fff60151b8b {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:18:21.340456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20df8d74c87143dd81dbaae76b3c2b62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94a935febe14fc5b0bda4ca88f78ca8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:21.350269 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ef67def3-df27-4351-809f-af63625fcd01 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 200 len: 1332 microversion: 2.1 time: 0.127261 Jul 31 01:18:21.350887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 490/981] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:21 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 1332 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:21.581103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:21.582805 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:21.593148 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:21.593932 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:21.595869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:21.596771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:21.819429 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:21.841194 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-05ae6b95-4930-47a2-9b7f-a7c53265be90 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.267462 Jul 31 01:18:21.842233 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 492/982] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:21 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:22.378913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:22.380125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:22.394231 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:22.395036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:22.396546 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:22.397400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:22.422874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-f86ca7f1-8ae2-4103-adc2-b65a6a7154bf req-fc2c9f72-cfcb-460f-95cf-857682053a54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d3c5a74-7a04-4f8b-a068-e68566b7d494", "name": "network-vif-plugged", "status": "completed", "tag": "55257163-7d49-4d57-9f0c-9a47106164ab"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:22.428433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b HTTP/1.1 Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:18:22.432145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:22.434517 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '29c5be43-d161-49e2-8214-6fff60151b8b'} Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:22.435471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:22.436868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:22.438072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:22.438072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-49657f83-617e-4b52-9598-21261bfeb736 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 404 len: 111 microversion: 2.1 time: 0.063999 Jul 31 01:18:22.438072 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 491/983] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:22 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:22.445118 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f86ca7f1-8ae2-4103-adc2-b65a6a7154bf req-fc2c9f72-cfcb-460f-95cf-857682053a54 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:22.446618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-f86ca7f1-8ae2-4103-adc2-b65a6a7154bf req-fc2c9f72-cfcb-460f-95cf-857682053a54 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:22.470172 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6d7cbcb7-2c41-4c5e-a489-bb3420f6f6fc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '.version_select at 0x7f3e7b625ca0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:22.476753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-f86ca7f1-8ae2-4103-adc2-b65a6a7154bf req-fc2c9f72-cfcb-460f-95cf-857682053a54 service nova] Creating event network-vif-plugged:55257163-7d49-4d57-9f0c-9a47106164ab for instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 on hv-cinder-maste Jul 31 01:18:22.484190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-f86ca7f1-8ae2-4103-adc2-b65a6a7154bf req-fc2c9f72-cfcb-460f-95cf-857682053a54 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065299 Jul 31 01:18:22.485342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 493/984] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:18:22.518942 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6d7cbcb7-2c41-4c5e-a489-bb3420f6f6fc tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1297203599" status: 202 len: 0 microversion: 2.1 time: 0.071976 Jul 31 01:18:22.520674 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 492/985] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Jul 31 01:18:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1297203599 => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:18:22.870883 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:22.873096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:22.882886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:22.884117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:22.886323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:22.887104 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:23.116779 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:23.133886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7aef8860-237d-451a-b6a7-422f51943b07 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.270435 Jul 31 01:18:23.135062 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 494/986] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:22 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:24.176586 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:24.177907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:24.194917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:24.195929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:24.197171 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:24.197821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:24.378414 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:24.388836 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-eae82152-515b-4d08-bd30-3a3d25042a33 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1266 microversion: 2.1 time: 0.220388 Jul 31 01:18:24.389804 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 493/987] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:24 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1266 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:24.752112 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:24.752736 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:24.768238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:24.768238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:24.768238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:24.768238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:24.783599 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b HTTP/1.1 Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:18:24.788318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.790963 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '29c5be43-d161-49e2-8214-6fff60151b8b'} Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:24.792486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.794935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.798275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:24.798275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ad36ba65-3dd5-43b7-82d7-3c67d38ba44c tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 404 len: 111 microversion: 2.1 time: 0.042299 Jul 31 01:18:24.798275 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 495/988] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:18:24 2022] DELETE /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:18:24.803213 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:24.803875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 29c5be43-d161-49e2-8214-6fff60151b8b] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:24.811706 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:24.812309 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:24.813620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:24.814351 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:24.830757 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b HTTP/1.1 Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:18:24.832769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.835641 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '29c5be43-d161-49e2-8214-6fff60151b8b'} Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:24.837078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. Jul 31 01:18:24.839481 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:24.841787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 29c5be43-d161-49e2-8214-6fff60151b8b could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:24.841787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5ddcd4a9-7a5e-45f1-bd19-d86f46205af5 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b" status: 404 len: 111 microversion: 2.1 time: 0.035287 Jul 31 01:18:24.841787 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 494/989] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:24 2022] GET /compute/v2.1/servers/29c5be43-d161-49e2-8214-6fff60151b8b => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:25.412829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:25.413578 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:25.421096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:25.421680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:25.422533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:25.423290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:25.592095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:25.602932 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-44f3f038-3515-4fd2-ab20-17df519d823a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1452 microversion: 2.1 time: 0.195246 Jul 31 01:18:25.603625 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 496/990] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:25 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1452 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:25.625017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:25.626073 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:25.634308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:25.634910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:25.636113 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:25.636559 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:25.824516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:25.833313 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-884a2a6e-f56a-4159-9560-7fd9cd47160b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1452 microversion: 2.1 time: 0.213057 Jul 31 01:18:25.833905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 495/991] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:25 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1452 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:26.029666 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:26.030602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:26.039182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.039571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.040528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.041234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.059318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 HTTP/1.1 Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:18:26.060979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.063735 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'd1cf1b41-8f07-4466-b084-79a321360da3'} Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:26.065722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.069491 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.074154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:26.074154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-aa8d3638-ef94-4a2e-a2b7-865aef73b48e tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 404 len: 111 microversion: 2.1 time: 0.036957 Jul 31 01:18:26.074154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 497/992] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:18:26 2022] DELETE /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:26.079055 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:26.080356 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: d1cf1b41-8f07-4466-b084-79a321360da3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:26.098341 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.099333 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.100728 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.101351 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.121287 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 HTTP/1.1 Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:18:26.123170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.126009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'd1cf1b41-8f07-4466-b084-79a321360da3'} Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:26.128259 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. Jul 31 01:18:26.131097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.133709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance d1cf1b41-8f07-4466-b084-79a321360da3 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:26.133709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-789a8ef2-4af3-4997-8811-ff8bcacc33b6 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3" status: 404 len: 111 microversion: 2.1 time: 0.056376 Jul 31 01:18:26.133709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 496/993] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:26 2022] GET /compute/v2.1/servers/d1cf1b41-8f07-4466-b084-79a321360da3 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:26.138227 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:26.138883 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:26.149419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.150210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.152165 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.152575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.185173 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf HTTP/1.1 Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:18:26.188598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.191423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '2b53247d-da0a-46f9-aa9e-f4d09342e5bf'} Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:26.192936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.197419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.199760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:26.199760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-11520856-3063-476e-9924-da6d0b632e10 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 404 len: 111 microversion: 2.1 time: 0.057898 Jul 31 01:18:26.199760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 498/994] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:18:26 2022] DELETE /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:26.206123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:26.207191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 2b53247d-da0a-46f9-aa9e-f4d09342e5bf] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:26.219042 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.219890 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.221902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:26.222946 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:26.257278 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf HTTP/1.1 Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:18:26.260169 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.262526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '2b53247d-da0a-46f9-aa9e-f4d09342e5bf'} Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:26.263408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. Jul 31 01:18:26.265821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:26.268107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 2b53247d-da0a-46f9-aa9e-f4d09342e5bf could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:26.268107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-648efe2b-9116-42bf-804a-15985a016e9b tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf" status: 404 len: 111 microversion: 2.1 time: 0.064037 Jul 31 01:18:26.268107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 497/995] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:26 2022] GET /compute/v2.1/servers/2b53247d-da0a-46f9-aa9e-f4d09342e5bf => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:27.480932 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:27.483289 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:27.487469 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=76 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:27.497803 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:27.498620 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:27.500104 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:27.500917 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:27.529002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 HTTP/1.1 Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:18:27.532246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.534385 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '7ffec4de-cf74-4e4f-acc9-ea172ae3e171'} Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:27.535143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.537145 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.538437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:27.538437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-dfdc3fa2-1ae2-4fd6-98fd-6023dc741fec tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "DELETE /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 404 len: 111 microversion: 2.1 time: 0.061387 Jul 31 01:18:27.538437 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 499/996] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:18:27 2022] DELETE /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:27.552936 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:27.554519 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] [instance: 7ffec4de-cf74-4e4f-acc9-ea172ae3e171] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:27.558089 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:27.567033 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:27.567736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:27.568700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:27.569252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:27.590598 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] HTTP exception thrown: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Request method failure captured: Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 HTTP/1.1 Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYBx1pTNfEPXL3EylxcnY7_TG8z4-dBGwEQVF9W8yhf14zl6hRgWzSYR3FStXlOb5mvixh2ucG9Pvd_qBgT5h308SNhE1juepV9I7c2aEuAMiJvppj92j1KojvQayun1uni97XPRg37aOmkM-5lXfoj7fLYDjp8qes0Rjmfi1Rrlxk Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:18:27.592456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94a935febe14fc5b0bda4ca88f78ca8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94a935febe14fc5b0bda4ca88f78ca8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: b94a935febe14fc5b0bda4ca88f78ca8 Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1868274518 Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 20df8d74c87143dd81dbaae76b3c2b62 Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeNegativeTest-1868274518-project Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.595191 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '7ffec4de-cf74-4e4f-acc9-ea172ae3e171'} Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:18:27.602502 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. Jul 31 01:18:27.605760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:18:27.608066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] Returning 404 to user: Instance 7ffec4de-cf74-4e4f-acc9-ea172ae3e171 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:18:27.608066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-89cc9c3e-50cf-48aa-9a69-cc508d913903 tempest-AttachVolumeNegativeTest-1868274518 tempest-AttachVolumeNegativeTest-1868274518-project] 10.222.0.42 "GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171" status: 404 len: 111 microversion: 2.1 time: 0.052947 Jul 31 01:18:27.608066 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 498/997] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:27 2022] GET /compute/v2.1/servers/7ffec4de-cf74-4e4f-acc9-ea172ae3e171 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:18:28.670905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-de4ec94f-206c-4669-8b62-3d527e087e27 req-8201a3da-d3be-4da4-9f46-0be2e16b2dfe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d3c5a74-7a04-4f8b-a068-e68566b7d494", "tag": "55257163-7d49-4d57-9f0c-9a47106164ab"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:28.691095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-de4ec94f-206c-4669-8b62-3d527e087e27 req-8201a3da-d3be-4da4-9f46-0be2e16b2dfe service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:28.691920 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-de4ec94f-206c-4669-8b62-3d527e087e27 req-8201a3da-d3be-4da4-9f46-0be2e16b2dfe service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:28.716117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-de4ec94f-206c-4669-8b62-3d527e087e27 req-8201a3da-d3be-4da4-9f46-0be2e16b2dfe service nova] Creating event network-changed:55257163-7d49-4d57-9f0c-9a47106164ab for instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 on hv-cinder-maste Jul 31 01:18:28.724621 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-de4ec94f-206c-4669-8b62-3d527e087e27 req-8201a3da-d3be-4da4-9f46-0be2e16b2dfe service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060703 Jul 31 01:18:28.726420 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 500/998] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:38.656584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-de636cea-b2bb-440a-b91d-d8b946470afe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "52173785-f5e2-4e5e-bfa0-3352d26e048e", "device": "/dev/sdb"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:38.661247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-de636cea-b2bb-440a-b91d-d8b946470afe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:38.664491 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=35 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:38.665291 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:38.674835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-de636cea-b2bb-440a-b91d-d8b946470afe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:38.675197 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-de636cea-b2bb-440a-b91d-d8b946470afe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:38.676522 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-de636cea-b2bb-440a-b91d-d8b946470afe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:38.677063 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-de636cea-b2bb-440a-b91d-d8b946470afe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:39.070669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-de636cea-b2bb-440a-b91d-d8b946470afe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "POST /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.419414 Jul 31 01:18:39.071463 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 499/999] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:18:38 2022] POST /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9/os-volume_attachments => generated 194 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:57.194997 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:57.196114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:57.197488 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:57.199225 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:57.201434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:57.204563 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=81 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:18:57.220528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:57.220981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:57.222594 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:57.226260 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:57.277470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:18:57.436402 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c29c0f00-e957-4821-a229-52b21cf2b60d tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "DELETE /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 204 len: 0 microversion: 2.1 time: 0.258066 Jul 31 01:18:57.437241 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 501/1000] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:18:57 2022] DELETE /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:18:57.445840 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-4e88e559-ae9c-4c48-a3bb-05f0d2bd40d7 req-02e98bf9-844e-467b-a966-48215ac7bbd3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69cf3658-0cb6-48b8-b152-72b915385fbb", "tag": "38f41703-e2b2-4f28-9dc2-9118c346cfba"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:18:57.448175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:57.449396 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:57.458071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:57.459139 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:57.460581 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:57.461228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:57.471529 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-4e88e559-ae9c-4c48-a3bb-05f0d2bd40d7 req-02e98bf9-844e-467b-a966-48215ac7bbd3 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:57.472103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-4e88e559-ae9c-4c48-a3bb-05f0d2bd40d7 req-02e98bf9-844e-467b-a966-48215ac7bbd3 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:57.488483 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-4e88e559-ae9c-4c48-a3bb-05f0d2bd40d7 req-02e98bf9-844e-467b-a966-48215ac7bbd3 service nova] Creating event network-changed:38f41703-e2b2-4f28-9dc2-9118c346cfba for instance 69cf3658-0cb6-48b8-b152-72b915385fbb on hv-cinder-maste Jul 31 01:18:57.494916 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-4e88e559-ae9c-4c48-a3bb-05f0d2bd40d7 req-02e98bf9-844e-467b-a966-48215ac7bbd3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.309356 Jul 31 01:18:57.496089 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 500/1001] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:18:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:18:57.632255 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:57.640658 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9fb85b8b-9efc-4ca8-a82c-8bb51a8e6b11 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.197356 Jul 31 01:18:57.641250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 502/1002] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:57 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:18:58.663167 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:58.663979 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:58.671469 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:58.671927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:58.672761 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:58.673199 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:58.850705 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:18:58.865821 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bca02db1-037b-43b7-9316-e7dbba6b953f tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.206763 Jul 31 01:18:58.866908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 501/1003] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:58 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:18:59.890125 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:18:59.890821 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:18:59.898669 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:59.899212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:18:59.900151 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:18:59.900682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:00.074150 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:00.088114 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-57d5414e-4c31-4b95-a276-08e928afab14 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.202737 Jul 31 01:19:00.089129 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 503/1004] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:18:59 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:01.120028 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:01.121063 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:01.132311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:01.133096 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:01.134534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:01.135317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:01.314250 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:01.323181 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8168b71c-0f71-454a-9b38-393a863caf7a tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.210349 Jul 31 01:19:01.324403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 502/1005] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:01 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:02.350387 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:02.351613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:02.364914 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:02.365752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:02.367200 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:02.368082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:02.531502 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:02.541607 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-555df94e-d72d-4907-9205-b69083d9f189 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.198451 Jul 31 01:19:02.542829 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 504/1006] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:02 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:03.572250 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:03.573575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:03.586715 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:03.587429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:03.589105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:03.589788 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:03.890905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:03.906306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-881ba9a0-645b-4565-ab00-752416056d2e tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1570 microversion: 2.1 time: 0.342834 Jul 31 01:19:03.907440 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 503/1007] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:03 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1570 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:04.171671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-194096ab-100d-42a9-bc48-4338938d682e req-70c74cea-0533-4a9c-be39-d8fd8f70d368 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cf3658-0cb6-48b8-b152-72b915385fbb", "name": "network-vif-deleted", "tag": "38f41703-e2b2-4f28-9dc2-9118c346cfba"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:19:04.183602 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-194096ab-100d-42a9-bc48-4338938d682e req-70c74cea-0533-4a9c-be39-d8fd8f70d368 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:04.184062 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-194096ab-100d-42a9-bc48-4338938d682e req-70c74cea-0533-4a9c-be39-d8fd8f70d368 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:04.209217 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-194096ab-100d-42a9-bc48-4338938d682e req-70c74cea-0533-4a9c-be39-d8fd8f70d368 service nova] Creating event network-vif-deleted:38f41703-e2b2-4f28-9dc2-9118c346cfba for instance 69cf3658-0cb6-48b8-b152-72b915385fbb on hv-cinder-maste Jul 31 01:19:04.216300 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-194096ab-100d-42a9-bc48-4338938d682e req-70c74cea-0533-4a9c-be39-d8fd8f70d368 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048500 Jul 31 01:19:04.217471 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 505/1008] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:19:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:19:04.942457 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:04.943094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:04.952784 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:04.953560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:04.954878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:04.955722 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:04.996910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:04.997354 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:04.997812 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lazy-loading 'fault' on Instance uuid 69cf3658-0cb6-48b8-b152-72b915385fbb {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:19:05.049100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb4bfd7ea6664747a6e0693a2fd11e3b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca9597665cd4068b0a111136d6e0489', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:05.057792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bbde79c7-cbe0-4ffb-93e8-514a5f631eb9 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 200 len: 1225 microversion: 2.1 time: 0.119360 Jul 31 01:19:05.058618 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 504/1009] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:04 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:06.086588 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:06.087634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] [instance: 69cf3658-0cb6-48b8-b152-72b915385fbb] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:06.095949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:06.096552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:06.097354 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:06.097793 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:06.117972 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] HTTP exception thrown: Instance 69cf3658-0cb6-48b8-b152-72b915385fbb could not be found. Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Request method failure captured: Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb HTTP/1.1 Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dccRYx3CdYAiqvFfVnt9Q84-TsSR6hGIFru1arE87jpjo2YAQVrqSDm4D6KX0KTs4wqd3kUAwx3Oq2NC7M2yefB_FSC8yd5KLIwzidcLsSqmpZd1tYTVq0Ie2kePpfRjpwCGfACSYNhodACsV8wPjlfCi7GoIF8kbnRACB7ctpv2AE Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 5ca9597665cd4068b0a111136d6e0489 Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-TestShelveInstance-880943367 Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:19:06.119708 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/5ca9597665cd4068b0a111136d6e0489"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/5ca9597665cd4068b0a111136d6e0489"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/5ca9597665cd4068b0a111136d6e0489"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_5ca9597665cd4068b0a111136d6e0489", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-TestShelveInstance-880943367 Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 5ca9597665cd4068b0a111136d6e0489 Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-TestShelveInstance-880943367 Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-TestShelveInstance-880943367-project Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: cb4bfd7ea6664747a6e0693a2fd11e3b Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-TestShelveInstance-880943367-project Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 69cf3658-0cb6-48b8-b152-72b915385fbb could not be found. Jul 31 01:19:06.122716 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '69cf3658-0cb6-48b8-b152-72b915385fbb'} Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 69cf3658-0cb6-48b8-b152-72b915385fbb could not be found. Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:19:06.124470 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 69cf3658-0cb6-48b8-b152-72b915385fbb could not be found. Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 69cf3658-0cb6-48b8-b152-72b915385fbb could not be found. Jul 31 01:19:06.128135 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:06.131609 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Returning 404 to user: Instance 69cf3658-0cb6-48b8-b152-72b915385fbb could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:19:06.131609 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0f046848-3cc4-4fc7-a0a4-8517ecbc3eee tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb" status: 404 len: 111 microversion: 2.1 time: 0.039517 Jul 31 01:19:06.131609 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 506/1010] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:06 2022] GET /compute/v2.1/servers/69cf3658-0cb6-48b8-b152-72b915385fbb => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:19:06.328909 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ef3c7eef-fa73-414c-85b9-8089d2cb3053 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] Calling method '.version_select at 0x7f3e7b6f2ca0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:06.361208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ef3c7eef-fa73-414c-85b9-8089d2cb3053 tempest-TestShelveInstance-880943367 tempest-TestShelveInstance-880943367-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1706218805" status: 202 len: 0 microversion: 2.1 time: 0.038759 Jul 31 01:19:06.362341 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 505/1011] 10.222.0.42 () {58 vars in 1254 bytes} [Sun Jul 31 01:19:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1706218805 => generated 0 bytes in 41 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:19:06.620732 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:06.621639 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:06.630228 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:06.630684 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:06.631509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:06.631962 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:06.746170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lazy-loading 'services' on Instance uuid 3978683e-cb02-4c81-b2b9-10544bb204a9 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:19:06.827883 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fac6061d-c105-48b7-a804-3a6ba717d31f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9/os-volume_attachments/52173785-f5e2-4e5e-bfa0-3352d26e048e" status: 202 len: 0 microversion: 2.1 time: 0.214869 Jul 31 01:19:06.831760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 507/1012] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:19:06 2022] DELETE /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9/os-volume_attachments/52173785-f5e2-4e5e-bfa0-3352d26e048e => generated 0 bytes in 217 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:19:07.995986 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:07.996873 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:08.009365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:08.010165 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:08.012141 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:08.012931 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:08.063940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:19:08.196562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-14a774e7-043c-4ad4-b4d7-ab0e7f3db339 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 204 len: 0 microversion: 2.1 time: 0.208561 Jul 31 01:19:08.197317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 506/1013] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:19:07 2022] DELETE /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:19:08.215605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:08.217020 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:08.229690 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:08.230626 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:08.232183 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:08.233099 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:08.493461 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:08.509403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1c6a431c-a558-4944-9c24-dd52727d73ef tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1513 microversion: 2.1 time: 0.303647 Jul 31 01:19:08.529064 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 508/1014] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:08 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1513 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:09.550415 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:09.551973 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:09.571043 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:09.572320 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:09.573994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:09.575023 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:09.796568 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:09.809995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a9831af3-23fc-4fef-8800-5d5a29d07403 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1513 microversion: 2.1 time: 0.268010 Jul 31 01:19:09.811214 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 507/1015] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:09 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1513 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:19:10.845978 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:10.846868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:10.863861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:10.865274 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:10.867801 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:10.868971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:11.010302 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:11.024982 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fb5da52a-0114-437e-8c98-ebf63af5ba2d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 200 len: 1471 microversion: 2.1 time: 0.186902 Jul 31 01:19:11.025877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 509/1016] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:10 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 1471 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:11.154480 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-7a5b15f5-8acb-4bc2-8fe7-5ab905af1972 req-57b22c0c-5ac9-4ba3-b841-70737b45e890 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3978683e-cb02-4c81-b2b9-10544bb204a9", "name": "network-vif-deleted", "tag": "3c5adaa8-afed-461f-9a44-0d7402226cd4"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:19:11.166314 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-7a5b15f5-8acb-4bc2-8fe7-5ab905af1972 req-57b22c0c-5ac9-4ba3-b841-70737b45e890 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:11.166782 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-7a5b15f5-8acb-4bc2-8fe7-5ab905af1972 req-57b22c0c-5ac9-4ba3-b841-70737b45e890 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:11.180446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-7a5b15f5-8acb-4bc2-8fe7-5ab905af1972 req-57b22c0c-5ac9-4ba3-b841-70737b45e890 service nova] Creating event network-vif-deleted:3c5adaa8-afed-461f-9a44-0d7402226cd4 for instance 3978683e-cb02-4c81-b2b9-10544bb204a9 on hv-cinder-maste Jul 31 01:19:11.186603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-7a5b15f5-8acb-4bc2-8fe7-5ab905af1972 req-57b22c0c-5ac9-4ba3-b841-70737b45e890 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.135681 Jul 31 01:19:11.187225 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 508/1017] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:19:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:19:12.062316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:12.062968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: 3978683e-cb02-4c81-b2b9-10544bb204a9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:12.070440 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:12.070891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:12.071750 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:12.072339 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:12.089647 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] HTTP exception thrown: Instance 3978683e-cb02-4c81-b2b9-10544bb204a9 could not be found. Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Request method failure captured: Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 HTTP/1.1 Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dg34xucaxRgMJ6tiePrlp2O_7g-18gSBh6QAfIGJ3IHvW4pOtQIguPZL7DnG7grUFXAU413RaWkMa92eUTudzO_ejBqC_TJvwDtAtVNXTfb-qjY1dVx5X69wgzr0DxBfDy2JIYsYcSG24Q8o8tTQQ6AY5k710m3ruQ-5zbbmsftDZI Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 0f2417d70b00404f8e37d52f694f9a41 Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2054618116 Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:19:12.091366 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/0f2417d70b00404f8e37d52f694f9a41"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/0f2417d70b00404f8e37d52f694f9a41"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/0f2417d70b00404f8e37d52f694f9a41"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_0f2417d70b00404f8e37d52f694f9a41", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-VolumesSnapshotTestJSON-2054618116 Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 0f2417d70b00404f8e37d52f694f9a41 Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2054618116 Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-VolumesSnapshotTestJSON-2054618116-project Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 943af9a17a554646808ac2f2598a9adf Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-VolumesSnapshotTestJSON-2054618116-project Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 3978683e-cb02-4c81-b2b9-10544bb204a9 could not be found. Jul 31 01:19:12.093634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '3978683e-cb02-4c81-b2b9-10544bb204a9'} Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3978683e-cb02-4c81-b2b9-10544bb204a9 could not be found. Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:19:12.094664 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3978683e-cb02-4c81-b2b9-10544bb204a9 could not be found. Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3978683e-cb02-4c81-b2b9-10544bb204a9 could not be found. Jul 31 01:19:12.096190 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:12.098359 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Returning 404 to user: Instance 3978683e-cb02-4c81-b2b9-10544bb204a9 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:19:12.098359 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e8ad3c37-4e7f-4e12-b34c-c15dc71b233b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9" status: 404 len: 111 microversion: 2.1 time: 0.036319 Jul 31 01:19:12.098359 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 510/1018] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:12 2022] GET /compute/v2.1/servers/3978683e-cb02-4c81-b2b9-10544bb204a9 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:19:22.372029 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=26,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:22.372807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=50,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:22.377519 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-840850122", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "networks": [{"uuid": "5461ee14-86f9-434e-a42e-9a54983e738c"}]}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:19:22.443757 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] validate_networks() for [('5461ee14-86f9-434e-a42e-9a54983e738c', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:19:22.989176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:19:22.989728 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:19:22.990389 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:19:22.991078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:19:22.991686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:19:23.003238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:23.003841 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:23.004812 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:23.005292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:23.024828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Getting quotas for project 0f2417d70b00404f8e37d52f694f9a41. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:19:23.029601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Getting quotas for user 943af9a17a554646808ac2f2598a9adf and project 0f2417d70b00404f8e37d52f694f9a41. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:19:23.041889 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:19:23.044871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:19:23.096557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-454623ca-5851-4845-8ea6-eb25fb626916 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.725773 Jul 31 01:19:23.097692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 509/1019] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:19:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 728 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:19:23.111333 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-396a34b8-8419-4aa8-a5cf-10a6ee62cdc8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:23.112267 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-396a34b8-8419-4aa8-a5cf-10a6ee62cdc8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:23.116257 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=25,nova_api:UPDATE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:23.117194 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=59,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:23.192013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-396a34b8-8419-4aa8-a5cf-10a6ee62cdc8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:23.201546 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-396a34b8-8419-4aa8-a5cf-10a6ee62cdc8 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1218 microversion: 2.1 time: 0.094907 Jul 31 01:19:23.202132 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 511/1020] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:23 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1218 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:24.225276 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:24.226321 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:24.235134 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:24.235724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:24.236622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:24.237176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:24.327135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:24.335661 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5204b8e5-b07a-4b11-ad4e-ec8c7677b48d tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1270 microversion: 2.1 time: 0.115798 Jul 31 01:19:24.336274 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 510/1021] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:24 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1270 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:25.358507 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:25.359975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:25.373094 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:25.373890 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:25.375308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:25.376148 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:25.667315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:25.675607 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4d7a3b78-b165-409a-80a2-e48eebe2c8fe tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1318 microversion: 2.1 time: 0.321167 Jul 31 01:19:25.676210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 512/1022] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:25 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1318 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:19:26.031273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-bfb110b6-ac7f-46d3-a739-b96460481dbc req-59de5655-2135-40c7-8293-a57746d7e600 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb02f95a-09fb-4e16-be5f-675332833ce3", "tag": "611e6d88-8dd4-4b26-b75b-0fcf9eeab253"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:19:26.050419 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-bfb110b6-ac7f-46d3-a739-b96460481dbc req-59de5655-2135-40c7-8293-a57746d7e600 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:26.051203 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-bfb110b6-ac7f-46d3-a739-b96460481dbc req-59de5655-2135-40c7-8293-a57746d7e600 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:26.066859 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-bfb110b6-ac7f-46d3-a739-b96460481dbc req-59de5655-2135-40c7-8293-a57746d7e600 service nova] Creating event network-changed:611e6d88-8dd4-4b26-b75b-0fcf9eeab253 for instance eb02f95a-09fb-4e16-be5f-675332833ce3 on hv-cinder-maste Jul 31 01:19:26.072592 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-bfb110b6-ac7f-46d3-a739-b96460481dbc req-59de5655-2135-40c7-8293-a57746d7e600 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046251 Jul 31 01:19:26.073198 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 511/1023] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:19:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:19:26.704841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:26.705487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:26.712266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:26.712724 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:26.713512 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:26.714036 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:26.878382 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:26.886562 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c46f36c6-07dc-4911-a2ca-2e41f693d9aa tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1318 microversion: 2.1 time: 0.185709 Jul 31 01:19:26.887250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 513/1024] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:26 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1318 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:27.909720 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:27.910429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:27.917844 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:27.918308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:27.919101 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:27.919572 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:28.106194 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:28.114008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-661ccfae-aa67-4053-99ca-9b0c6b38cc15 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1318 microversion: 2.1 time: 0.209323 Jul 31 01:19:28.114596 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 512/1025] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:27 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1318 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:29.142216 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:29.142955 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:29.150542 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:29.150998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:29.151845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:29.152293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:29.428853 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:29.444256 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-efc416ca-bd23-4198-b013-bdc836687d03 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1318 microversion: 2.1 time: 0.305368 Jul 31 01:19:29.445285 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 514/1026] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:29 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1318 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:30.468400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:30.469593 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:30.482258 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:30.483063 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:30.484698 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:30.485495 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:30.713041 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:30.733415 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-74fe7bb6-5412-4d94-b273-714994a8321b tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1318 microversion: 2.1 time: 0.268932 Jul 31 01:19:30.747595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 513/1027] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:30 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1318 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:19:30.886927 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-b742f948-0572-4ca9-b00e-525c406688e7 req-e20063d6-7665-4ed9-bdf5-021bc3212c67 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb02f95a-09fb-4e16-be5f-675332833ce3", "name": "network-vif-plugged", "status": "completed", "tag": "611e6d88-8dd4-4b26-b75b-0fcf9eeab253"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:19:30.904295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b742f948-0572-4ca9-b00e-525c406688e7 req-e20063d6-7665-4ed9-bdf5-021bc3212c67 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:30.905078 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-b742f948-0572-4ca9-b00e-525c406688e7 req-e20063d6-7665-4ed9-bdf5-021bc3212c67 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:31.141654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-b742f948-0572-4ca9-b00e-525c406688e7 req-e20063d6-7665-4ed9-bdf5-021bc3212c67 service nova] Creating event network-vif-plugged:611e6d88-8dd4-4b26-b75b-0fcf9eeab253 for instance eb02f95a-09fb-4e16-be5f-675332833ce3 on hv-cinder-maste Jul 31 01:19:31.149552 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-b742f948-0572-4ca9-b00e-525c406688e7 req-e20063d6-7665-4ed9-bdf5-021bc3212c67 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.492629 Jul 31 01:19:31.150621 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 515/1028] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:19:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:19:31.771876 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:31.773245 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:31.784690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:31.785234 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:31.786086 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:31.786584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:31.946267 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:31.954928 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-93c233eb-b5a5-4a26-9335-e0d9924cc463 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1318 microversion: 2.1 time: 0.191168 Jul 31 01:19:31.955655 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 514/1029] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:31 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1318 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:32.977527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:32.978348 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:32.986372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:32.986822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:32.987686 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:32.988140 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:33.180825 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:33.189546 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-13a133f1-1b1c-4fae-a54c-1468c0d7c9e4 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1318 microversion: 2.1 time: 0.217001 Jul 31 01:19:33.190175 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 516/1030] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:32 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1318 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:34.225350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:34.226103 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:34.228524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:34.229366 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:34.235800 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:34.236254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:34.237044 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:34.237480 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:34.389544 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:34.411311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b1de5579-e507-4158-a541-b8b2f5cb822e tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1506 microversion: 2.1 time: 0.189764 Jul 31 01:19:34.412087 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 515/1031] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:34 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1506 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:34.438758 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-96eaadf4-948a-4912-92e9-1d55e0ec5b79 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b3467a5a-3ba9-4628-b374-d7dad5baee1a", "device": "/dev/sdb"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:19:34.441869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-96eaadf4-948a-4912-92e9-1d55e0ec5b79 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:34.444013 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:34.450677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96eaadf4-948a-4912-92e9-1d55e0ec5b79 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:34.451136 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96eaadf4-948a-4912-92e9-1d55e0ec5b79 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:34.451961 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96eaadf4-948a-4912-92e9-1d55e0ec5b79 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:34.452406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-96eaadf4-948a-4912-92e9-1d55e0ec5b79 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:34.768126 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-96eaadf4-948a-4912-92e9-1d55e0ec5b79 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "POST /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.333138 Jul 31 01:19:34.772229 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 517/1032] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:19:34 2022] POST /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3/os-volume_attachments => generated 194 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:19:37.968691 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1a55c2a9-4018-4bf9-ab51-eccadd596a7b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:37.971132 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1a55c2a9-4018-4bf9-ab51-eccadd596a7b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:37.979337 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1a55c2a9-4018-4bf9-ab51-eccadd596a7b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:37.979993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1a55c2a9-4018-4bf9-ab51-eccadd596a7b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:37.980886 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1a55c2a9-4018-4bf9-ab51-eccadd596a7b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:37.981468 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1a55c2a9-4018-4bf9-ab51-eccadd596a7b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:38.017664 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1a55c2a9-4018-4bf9-ab51-eccadd596a7b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.053829 Jul 31 01:19:38.018256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 516/1033] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Jul 31 01:19:37 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:19:38.114149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:38.114854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:38.122200 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:38.122652 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:38.123493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:38.123935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:38.155084 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:19:38.260034 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-9e7b84e2-e00e-4f42-bd9e-80e0309640da tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 204 len: 0 microversion: 2.20 time: 0.150974 Jul 31 01:19:38.261687 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 518/1034] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Jul 31 01:19:38 2022] DELETE /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 31 01:19:38.270650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:38.271347 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:38.278121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:38.278569 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:38.279345 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:38.279964 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:38.425658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:38.436358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:38.438100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:38.439336 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a766edaa-ccb9-43c0-aa7a-082bc87ae590 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.172887 Jul 31 01:19:38.440246 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 517/1035] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:19:38 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:19:39.464952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:39.465698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:39.472768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:39.473240 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:39.474058 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:39.474586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:39.646096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:39.655653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:39.657683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:39.659150 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ad0610d6-990e-4e0d-95f8-02d74b83c002 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.198658 Jul 31 01:19:39.659881 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 519/1036] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:19:39 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 31 01:19:40.688647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:40.689752 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:40.702476 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:40.703303 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:40.704690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:40.705518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:40.915534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:40.933834 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:40.937083 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:40.939619 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-066d6819-5d23-4291-a738-faea27a8ae9c tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1723 microversion: 2.20 time: 0.258836 Jul 31 01:19:40.940627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 518/1037] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:19:40 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1723 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:19:41.391654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-1741303b-6361-45c0-98ab-25f6497339a1 req-7c748f2b-1669-460a-9320-774707e620af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c03b1463-390f-430b-9bdf-64322c030504", "name": "network-vif-deleted", "tag": "7d683e21-e88a-4b21-ba07-f00c5ff5031e"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:19:41.409177 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1741303b-6361-45c0-98ab-25f6497339a1 req-7c748f2b-1669-460a-9320-774707e620af service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:41.409768 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-1741303b-6361-45c0-98ab-25f6497339a1 req-7c748f2b-1669-460a-9320-774707e620af service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:41.426392 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-1741303b-6361-45c0-98ab-25f6497339a1 req-7c748f2b-1669-460a-9320-774707e620af service nova] Creating event network-vif-deleted:7d683e21-e88a-4b21-ba07-f00c5ff5031e for instance c03b1463-390f-430b-9bdf-64322c030504 on hv-cinder-maste Jul 31 01:19:41.432750 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-1741303b-6361-45c0-98ab-25f6497339a1 req-7c748f2b-1669-460a-9320-774707e620af service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046235 Jul 31 01:19:41.433423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 520/1038] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:19:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:19:41.963995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:41.964934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:41.972616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:41.972968 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:41.973824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:41.974260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:42.010658 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:42.011743 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:42.012200 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lazy-loading 'fault' on Instance uuid c03b1463-390f-430b-9bdf-64322c030504 {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:19:42.061661 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:42.071682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:42.073060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '56732ebe8ee24f3f9ce8f87746e8b161', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '80ea98b340d94181b2193f4a79905988', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:42.075065 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-883ceacc-0a11-44a6-bb0d-db3a8c718b31 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 200 len: 1370 microversion: 2.20 time: 0.115456 Jul 31 01:19:42.076106 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 519/1039] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:19:41 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 1370 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 31 01:19:43.099868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:43.100582 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:43.107711 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:43.108166 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:43.108969 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:43.109407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:43.127475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] HTTP exception thrown: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Request method failure captured: Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 HTTP/1.1 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYE5CsTG8XAGRyb7_DMNnr9aW5_pey_uEysBzVl7GCzZbKB1Tms3eNompJRXqRjZEel38CmpCHlWf15YLg0A5CerD_bRG6LtuidRf7CvO4XxrxqeneoQvqNyWZoFZeZUbgGKZ07QkooErBFk3wzINIfyHq-pgQ54Be66TzjO0BDud4 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Openstack-Nova-Api-Version: 2.20 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:19:43.129170 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/80ea98b340d94181b2193f4a79905988"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_80ea98b340d94181b2193f4a79905988", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 56732ebe8ee24f3f9ce8f87746e8b161 Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:43.132085 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'c03b1463-390f-430b-9bdf-64322c030504'} Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:19:43.133504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:43.136605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:43.139412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Returning 404 to user: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:19:43.139412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6afed523-e482-4e06-b716-f55b298c83c3 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 404 len: 111 microversion: 2.20 time: 0.035816 Jul 31 01:19:43.139412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 521/1040] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:19:43 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 31 01:19:43.143876 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-27baccb3-ded4-442b-b446-765c3c4e8492 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '.version_select at 0x7f3e7b59b160>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:43.166292 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-27baccb3-ded4-442b-b446-765c3c4e8492 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-534147176" status: 204 len: 0 microversion: 2.20 time: 0.026736 Jul 31 01:19:43.167291 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 520/1041] 10.222.0.42 () {60 vars in 1259 bytes} [Sun Jul 31 01:19:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-534147176 => generated 0 bytes in 29 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 31 01:19:48.207992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:48.208610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:48.217558 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:48.218290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:48.219633 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:48.220346 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:48.250300 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] HTTP exception thrown: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Request method failure captured: Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 HTTP/1.1 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYE5CsTG8XAGRyb7_DMNnr9aW5_pey_uEysBzVl7GCzZbKB1Tms3eNompJRXqRjZEel38CmpCHlWf15YLg0A5CerD_bRG6LtuidRf7CvO4XxrxqeneoQvqNyWZoFZeZUbgGKZ07QkooErBFk3wzINIfyHq-pgQ54Be66TzjO0BDud4 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Openstack-Nova-Api-Version: 2.20 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:19:48.253319 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/80ea98b340d94181b2193f4a79905988"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_80ea98b340d94181b2193f4a79905988", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 56732ebe8ee24f3f9ce8f87746e8b161 Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.255047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'c03b1463-390f-430b-9bdf-64322c030504'} Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:19:48.255855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.257077 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.258354 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Returning 404 to user: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:19:48.259649 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f8a0b2a3-bc24-4c0d-b055-c7d5dff72bb5 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 404 len: 111 microversion: 2.20 time: 0.055133 Jul 31 01:19:48.260740 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 522/1042] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Jul 31 01:19:48 2022] DELETE /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 31 01:19:48.271938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:48.272603 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: c03b1463-390f-430b-9bdf-64322c030504] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:48.282990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:48.283627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:48.284553 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:48.285048 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:48.303094 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] HTTP exception thrown: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Request method failure captured: Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 HTTP/1.1 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYE5CsTG8XAGRyb7_DMNnr9aW5_pey_uEysBzVl7GCzZbKB1Tms3eNompJRXqRjZEel38CmpCHlWf15YLg0A5CerD_bRG6LtuidRf7CvO4XxrxqeneoQvqNyWZoFZeZUbgGKZ07QkooErBFk3wzINIfyHq-pgQ54Be66TzjO0BDud4 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Openstack-Nova-Api-Version: 2.20 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:19:48.305228 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/80ea98b340d94181b2193f4a79905988"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_80ea98b340d94181b2193f4a79905988", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 56732ebe8ee24f3f9ce8f87746e8b161 Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.307105 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'c03b1463-390f-430b-9bdf-64322c030504'} Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:19:48.308423 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. Jul 31 01:19:48.310453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:48.312022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Returning 404 to user: Instance c03b1463-390f-430b-9bdf-64322c030504 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:19:48.312022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e0494227-8bcc-4b3d-8c85-bb550ae0b3b8 tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504" status: 404 len: 111 microversion: 2.20 time: 0.039867 Jul 31 01:19:48.312022 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 521/1043] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:19:48 2022] GET /compute/v2.1/servers/c03b1463-390f-430b-9bdf-64322c030504 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 31 01:19:53.769215 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:53.769834 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:53.772541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=32,nova_api:UPDATE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:53.780891 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:53.781554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:53.782503 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:53.783047 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:53.787550 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:53.803673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] HTTP exception thrown: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Request method failure captured: Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: DELETE /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 HTTP/1.1 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dYE5CsTG8XAGRyb7_DMNnr9aW5_pey_uEysBzVl7GCzZbKB1Tms3eNompJRXqRjZEel38CmpCHlWf15YLg0A5CerD_bRG6LtuidRf7CvO4XxrxqeneoQvqNyWZoFZeZUbgGKZ07QkooErBFk3wzINIfyHq-pgQ54Be66TzjO0BDud4 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Openstack-Nova-Api-Version: 2.20 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:19:53.805541 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/80ea98b340d94181b2193f4a79905988"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_80ea98b340d94181b2193f4a79905988", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 56732ebe8ee24f3f9ce8f87746e8b161 Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.809323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'ef19c2df-372c-4c77-ad6c-ada1cd419a14'} Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:19:53.810937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.813363 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.815878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Returning 404 to user: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:19:53.815878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8a0c839e-26ea-44ce-a950-7a77e535777b tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 404 len: 111 microversion: 2.20 time: 0.042464 Jul 31 01:19:53.815878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 523/1044] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Jul 31 01:19:53 2022] DELETE /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 31 01:19:53.820681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:53.821338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] [instance: ef19c2df-372c-4c77-ad6c-ada1cd419a14] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:53.824146 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:53.830619 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:53.831220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:53.832145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:53.832845 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:53.837245 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=63 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:19:53.852350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] HTTP exception thrown: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Request method failure captured: Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 HTTP/1.1 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dYE5CsTG8XAGRyb7_DMNnr9aW5_pey_uEysBzVl7GCzZbKB1Tms3eNompJRXqRjZEel38CmpCHlWf15YLg0A5CerD_bRG6LtuidRf7CvO4XxrxqeneoQvqNyWZoFZeZUbgGKZ07QkooErBFk3wzINIfyHq-pgQ54Be66TzjO0BDud4 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Openstack-Nova-Api-Version: 2.20 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:19:53.853969 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/80ea98b340d94181b2193f4a79905988"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/80ea98b340d94181b2193f4a79905988"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_80ea98b340d94181b2193f4a79905988", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 80ea98b340d94181b2193f4a79905988 Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1101964941 Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 56732ebe8ee24f3f9ce8f87746e8b161 Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1101964941-project Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.856625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'ef19c2df-372c-4c77-ad6c-ada1cd419a14'} Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:19:53.858140 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. Jul 31 01:19:53.860492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:19:53.862872 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] Returning 404 to user: Instance ef19c2df-372c-4c77-ad6c-ada1cd419a14 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:19:53.862872 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-99f0ce8c-b5ef-49ec-bbb4-dfeb568e73ce tempest-AttachVolumeShelveTestJSON-1101964941 tempest-AttachVolumeShelveTestJSON-1101964941-project] 10.222.0.42 "GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14" status: 404 len: 111 microversion: 2.20 time: 0.040666 Jul 31 01:19:53.862872 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 522/1045] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Jul 31 01:19:53 2022] GET /compute/v2.1/servers/ef19c2df-372c-4c77-ad6c-ada1cd419a14 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 31 01:19:56.639253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:56.640220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:56.652924 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:56.653672 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:56.655019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:56.655730 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:56.790681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lazy-loading 'services' on Instance uuid eb02f95a-09fb-4e16-be5f-675332833ce3 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:19:56.891702 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4b5a1a91-52f8-4e7e-982b-7f05859ac8b7 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "DELETE /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3/os-volume_attachments/b3467a5a-3ba9-4628-b374-d7dad5baee1a" status: 202 len: 0 microversion: 2.1 time: 0.259178 Jul 31 01:19:56.893044 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 524/1046] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:19:56 2022] DELETE /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3/os-volume_attachments/b3467a5a-3ba9-4628-b374-d7dad5baee1a => generated 0 bytes in 262 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:19:58.124185 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:58.126609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:58.148616 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:58.149819 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:58.153254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:58.154505 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:58.215609 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:19:58.360801 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cf20368d-64f2-481b-954e-a9245b94471f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "DELETE /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 204 len: 0 microversion: 2.1 time: 0.244600 Jul 31 01:19:58.361841 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 523/1047] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:19:58 2022] DELETE /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 31 01:19:58.378103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:58.379609 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:58.394238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:58.395784 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:58.398587 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:58.399638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:58.488822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:19:58.489082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:19:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 (ip 10.222.0.42) !!! Jul 31 01:19:58.634119 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:58.646516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b9531893-d452-4578-a00c-1376f381d54f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1512 microversion: 2.1 time: 0.277378 Jul 31 01:19:58.647266 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 525/1048] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:58 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1512 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:19:59.257442 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:19:59.257667 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:19:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:19:59.677910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:19:59.679135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:19:59.690220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:59.690934 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:59.692554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:19:59.693232 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:19:59.906353 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:19:59.915318 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0ee24f14-368b-4a03-8350-e6da4442042f tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1512 microversion: 2.1 time: 0.246165 Jul 31 01:19:59.915944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 524/1049] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:19:59 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1512 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:00.954003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:00.956695 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:00.964915 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:00.966272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:00.967388 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:00.968727 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:00.990745 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-4e9ac514-0db4-4f5f-abd5-4286b9d97334 req-c55057de-67ab-4e79-a9f4-251d2a5e0312 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb02f95a-09fb-4e16-be5f-675332833ce3", "name": "network-vif-deleted", "tag": "611e6d88-8dd4-4b26-b75b-0fcf9eeab253"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:20:01.008583 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-4e9ac514-0db4-4f5f-abd5-4286b9d97334 req-c55057de-67ab-4e79-a9f4-251d2a5e0312 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:01.009402 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-4e9ac514-0db4-4f5f-abd5-4286b9d97334 req-c55057de-67ab-4e79-a9f4-251d2a5e0312 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:01.041958 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-4e9ac514-0db4-4f5f-abd5-4286b9d97334 req-c55057de-67ab-4e79-a9f4-251d2a5e0312 service nova] Creating event network-vif-deleted:611e6d88-8dd4-4b26-b75b-0fcf9eeab253 for instance eb02f95a-09fb-4e16-be5f-675332833ce3 on hv-cinder-maste Jul 31 01:20:01.051682 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-4e9ac514-0db4-4f5f-abd5-4286b9d97334 req-c55057de-67ab-4e79-a9f4-251d2a5e0312 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064196 Jul 31 01:20:01.057131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 525/1050] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:20:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:20:01.100422 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '943af9a17a554646808ac2f2598a9adf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0f2417d70b00404f8e37d52f694f9a41', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:01.127885 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-00658ead-2574-43f9-96ea-c9d51fabb874 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 200 len: 1470 microversion: 2.1 time: 0.180106 Jul 31 01:20:01.129386 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 526/1051] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:00 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 1470 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:20:02.174049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:02.177584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] [instance: eb02f95a-09fb-4e16-be5f-675332833ce3] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:02.197612 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:02.200816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:02.202625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:02.203257 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:02.237799 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] HTTP exception thrown: Instance eb02f95a-09fb-4e16-be5f-675332833ce3 could not be found. Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Request method failure captured: Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 HTTP/1.1 Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dg34xucaxRgMJ6tiePrlp2O_7g-18gSBh6QAfIGJ3IHvW4pOtQIguPZL7DnG7grUFXAU413RaWkMa92eUTudzO_ejBqC_TJvwDtAtVNXTfb-qjY1dVx5X69wgzr0DxBfDy2JIYsYcSG24Q8o8tTQQ6AY5k710m3ruQ-5zbbmsftDZI Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 0f2417d70b00404f8e37d52f694f9a41 Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2054618116 Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:20:02.240534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/0f2417d70b00404f8e37d52f694f9a41"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/0f2417d70b00404f8e37d52f694f9a41"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/0f2417d70b00404f8e37d52f694f9a41"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_0f2417d70b00404f8e37d52f694f9a41", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-VolumesSnapshotTestJSON-2054618116 Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 0f2417d70b00404f8e37d52f694f9a41 Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2054618116 Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-VolumesSnapshotTestJSON-2054618116-project Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 943af9a17a554646808ac2f2598a9adf Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-VolumesSnapshotTestJSON-2054618116-project Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance eb02f95a-09fb-4e16-be5f-675332833ce3 could not be found. Jul 31 01:20:02.242549 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'eb02f95a-09fb-4e16-be5f-675332833ce3'} Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance eb02f95a-09fb-4e16-be5f-675332833ce3 could not be found. Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:20:02.243681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance eb02f95a-09fb-4e16-be5f-675332833ce3 could not be found. Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance eb02f95a-09fb-4e16-be5f-675332833ce3 could not be found. Jul 31 01:20:02.246273 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:20:02.250827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] Returning 404 to user: Instance eb02f95a-09fb-4e16-be5f-675332833ce3 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:20:02.252765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-db45a7ae-b7e9-4fa2-b087-3cf884170919 tempest-VolumesSnapshotTestJSON-2054618116 tempest-VolumesSnapshotTestJSON-2054618116-project] 10.222.0.42 "GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3" status: 404 len: 111 microversion: 2.1 time: 0.096694 Jul 31 01:20:02.254528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 526/1052] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:02 2022] GET /compute/v2.1/servers/eb02f95a-09fb-4e16-be5f-675332833ce3 => generated 111 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:20:07.249362 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-198bf746-1990-4d0d-a8e0-b378eb0e81e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'create', calling method: .version_select at 0x7f3e7b531430>, body: {"keypair": {"name": "tempest-keypair-1558543546", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCqU11G1+JcDfPc+dKS4MAyKbeoiudc5zpzrfP6/Ww1qAMfHxMJf31uA3c6NRBkLjNpfQOLsHmvEMM7vKVq5iltAzCFpUO80lSJeSozQl6rkJmcOO9pLV7GSVeHdsjYLWA=="}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:20:07.273209 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-198bf746-1990-4d0d-a8e0-b378eb0e81e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for project 17c952edcc5440989be309944a4c0b00. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:20:07.283117 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-198bf746-1990-4d0d-a8e0-b378eb0e81e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for user 64fea53260874eae8c13416c97cfa4c7 and project 17c952edcc5440989be309944a4c0b00. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:20:07.329158 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-198bf746-1990-4d0d-a8e0-b378eb0e81e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.254002 Jul 31 01:20:07.332526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 527/1053] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:20:07 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:20:09.459208 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-867192278", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "164a7c25-e57c-4866-8228-8acca369f667"}], "security_groups": [{"name": "tempest-securitygroup--1186890247"}], "key_name": "tempest-keypair-1558543546", "user_data": "}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:20:09.600991 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] validate_networks() for [('164a7c25-e57c-4866-8228-8acca369f667', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:20:10.296631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:20:10.297381 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:20:10.298382 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:20:10.298989 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:20:10.300722 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:20:10.315785 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:10.316492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:10.317546 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:10.318093 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:10.344651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for project 17c952edcc5440989be309944a4c0b00. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:20:10.351409 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for user 64fea53260874eae8c13416c97cfa4c7 and project 17c952edcc5440989be309944a4c0b00. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:20:10.373028 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:20:10.376309 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:20:10.450017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8cd93e4c-e520-4a6a-ad46-1003ee60f932 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.999015 Jul 31 01:20:10.451296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 527/1054] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:20:09 2022] POST /compute/v2.1/servers => generated 402 bytes in 1001 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 31 01:20:10.462688 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-596d3f12-d24b-4333-bd7a-3d3dcd5a49e3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:10.463498 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-596d3f12-d24b-4333-bd7a-3d3dcd5a49e3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:10.529410 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-596d3f12-d24b-4333-bd7a-3d3dcd5a49e3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:10.538268 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-596d3f12-d24b-4333-bd7a-3d3dcd5a49e3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1237 microversion: 2.1 time: 0.080063 Jul 31 01:20:10.539163 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 528/1055] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:10 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1237 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:11.569769 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:11.570594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:11.577539 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:11.578057 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:11.578892 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:11.579418 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:11.696519 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:11.706578 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-08f9a57e-4c40-404c-8b06-3166cb5d06fd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1289 microversion: 2.1 time: 0.145818 Jul 31 01:20:11.707543 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 528/1056] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:11 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1289 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:12.747315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:12.748821 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:12.752202 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=22,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:20:12.766931 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:12.767846 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:12.769254 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:12.770043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:12.936096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:12.947577 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e1d461d5-839e-4766-bb1e-1344712081ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1295 microversion: 2.1 time: 0.208329 Jul 31 01:20:12.960212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 529/1057] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:12 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1295 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:13.976332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:13.977566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:13.988128 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:13.989261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:13.990724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:13.991666 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:14.235181 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:14.252252 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b8bbaf2f-c3e1-48f4-8c6b-cb1bfe0dd3a3 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.282057 Jul 31 01:20:14.254839 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 529/1058] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:13 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:14.476572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-d42ea198-0cae-4fea-b5d4-0e67a5b01ace req-1a0dcf15-2151-4919-b2cd-56014db23d99 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ca751a3a-0157-4e91-b8ca-f6594fca8eb9", "tag": "2bf3dbb7-a8f6-41f1-92f3-95c49ff50266"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:20:14.490904 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-d42ea198-0cae-4fea-b5d4-0e67a5b01ace req-1a0dcf15-2151-4919-b2cd-56014db23d99 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:14.491523 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-d42ea198-0cae-4fea-b5d4-0e67a5b01ace req-1a0dcf15-2151-4919-b2cd-56014db23d99 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:14.506318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-d42ea198-0cae-4fea-b5d4-0e67a5b01ace req-1a0dcf15-2151-4919-b2cd-56014db23d99 service nova] Creating event network-changed:2bf3dbb7-a8f6-41f1-92f3-95c49ff50266 for instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 on hv-cinder-maste Jul 31 01:20:14.513473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-d42ea198-0cae-4fea-b5d4-0e67a5b01ace req-1a0dcf15-2151-4919-b2cd-56014db23d99 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042184 Jul 31 01:20:14.514367 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 530/1059] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:20:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:20:15.285427 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:15.286162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:15.297648 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:15.298358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:15.299943 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:15.300702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:15.487717 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:15.496160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cfafb910-7150-4d9c-bb5e-7317f944c3f7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.215233 Jul 31 01:20:15.496830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 530/1060] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:15 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:16.518473 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:16.519575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:16.526771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:16.527226 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:16.528055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:16.528492 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:16.696407 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:16.704975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-82071b01-78a7-4526-b57a-71315248d449 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.191053 Jul 31 01:20:16.705589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 531/1061] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:16 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:20:17.738595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:17.739663 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:17.749520 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:17.750280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:17.751589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:17.752126 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:17.945881 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:17.962681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d30efb9b-2994-4e43-87be-b770643a96ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.228759 Jul 31 01:20:17.967593 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 531/1062] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:17 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:18.998820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:18.999994 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:19.008883 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:19.009618 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:19.011489 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:19.012199 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:19.215115 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:19.225230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7c6731e0-d20e-4f55-8123-4b430c1af311 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.230828 Jul 31 01:20:19.226123 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 532/1063] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:18 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:20.265349 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:20.266028 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:20.273667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:20.274120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:20.274915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:20.275383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:20.328016 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:20:20.361679 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:20:20.441941 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:20.450511 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3170adcd-8d79-4819-94f1-6b16e009aa58 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.189375 Jul 31 01:20:20.451121 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 532/1064] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:20 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:21.472291 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:21.472885 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:21.475644 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=6 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:20:21.482742 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:21.483290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:21.485076 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:21.485536 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:21.640760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:21.652049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-09afe3dc-bb79-4d8f-9362-d02deef511bb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.183231 Jul 31 01:20:21.652681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 533/1065] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:21 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:22.687497 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:22.688117 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:22.695554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:22.696095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:22.697100 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:22.697589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:22.891830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:22.906561 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-16880f98-565f-43f5-8071-5ef5d7d24ff9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.224321 Jul 31 01:20:22.907636 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 533/1066] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:22 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:23.940328 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:23.941196 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:23.943490 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:20:23.950037 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:23.950499 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:23.951309 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:23.951790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:24.131976 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:24.142588 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-daaaeca4-2660-454d-b229-ad609b9eed9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.205904 Jul 31 01:20:24.156272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 534/1067] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:23 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:24.403282 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-de855d55-9d57-4a06-885a-844b3e2d60db req-c9f486f6-57b6-4825-ad1c-22e954ddad64 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ca751a3a-0157-4e91-b8ca-f6594fca8eb9", "name": "network-vif-plugged", "status": "completed", "tag": "2bf3dbb7-a8f6-41f1-92f3-95c49ff50266"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:20:24.411312 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:20:24.416899 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-de855d55-9d57-4a06-885a-844b3e2d60db req-c9f486f6-57b6-4825-ad1c-22e954ddad64 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:24.417350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-de855d55-9d57-4a06-885a-844b3e2d60db req-c9f486f6-57b6-4825-ad1c-22e954ddad64 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:24.423400 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:20:24.434486 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-de855d55-9d57-4a06-885a-844b3e2d60db req-c9f486f6-57b6-4825-ad1c-22e954ddad64 service nova] Creating event network-vif-plugged:2bf3dbb7-a8f6-41f1-92f3-95c49ff50266 for instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 on hv-cinder-maste Jul 31 01:20:24.440684 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-de855d55-9d57-4a06-885a-844b3e2d60db req-c9f486f6-57b6-4825-ad1c-22e954ddad64 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044134 Jul 31 01:20:24.441755 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 534/1068] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:20:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:20:25.168454 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:25.169211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:25.176535 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:25.177056 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:25.178014 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:25.178507 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:25.356055 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:25.368951 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3f6559a2-acaf-4980-a993-d9406b52bcb9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.204385 Jul 31 01:20:25.369968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 535/1069] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:25 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:26.399323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:26.400623 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:26.414479 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:26.415266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:26.416708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:26.417492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:26.646401 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:26.658995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b8354d46-d367-4dbb-a2b8-ccd091188a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1363 microversion: 2.1 time: 0.269757 Jul 31 01:20:26.660332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 535/1070] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:26 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1363 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:20:27.682700 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:27.683500 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:27.691300 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:27.691947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:27.692810 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:27.693308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:27.873075 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:20:27.883406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b0f21b08-2c3a-4b8d-8b1b-a2b0edfac656 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1550 microversion: 2.1 time: 0.204814 Jul 31 01:20:27.884236 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 536/1071] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:20:27 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1550 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:20:27.918535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3e7b0965-babc-4927-b633-bbd820b8aa08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:20:27.919097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3e7b0965-babc-4927-b633-bbd820b8aa08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:20:27.926545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e7b0965-babc-4927-b633-bbd820b8aa08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:27.927015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e7b0965-babc-4927-b633-bbd820b8aa08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:27.927864 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e7b0965-babc-4927-b633-bbd820b8aa08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:27.928408 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3e7b0965-babc-4927-b633-bbd820b8aa08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:28.058113 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3e7b0965-babc-4927-b633-bbd820b8aa08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.143692 Jul 31 01:20:28.060034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 536/1072] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:20:27 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-interface => generated 285 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:20:29.709793 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-8fafec0f-a698-471f-b490-df27834c27c1 req-360557ea-abb3-4252-ae82-275f1e12ec3f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ca751a3a-0157-4e91-b8ca-f6594fca8eb9", "tag": "2bf3dbb7-a8f6-41f1-92f3-95c49ff50266"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:20:29.734513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-8fafec0f-a698-471f-b490-df27834c27c1 req-360557ea-abb3-4252-ae82-275f1e12ec3f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:20:29.735841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-8fafec0f-a698-471f-b490-df27834c27c1 req-360557ea-abb3-4252-ae82-275f1e12ec3f service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:20:29.756990 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-8fafec0f-a698-471f-b490-df27834c27c1 req-360557ea-abb3-4252-ae82-275f1e12ec3f service nova] Creating event network-changed:2bf3dbb7-a8f6-41f1-92f3-95c49ff50266 for instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 on hv-cinder-maste Jul 31 01:20:29.768755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-8fafec0f-a698-471f-b490-df27834c27c1 req-360557ea-abb3-4252-ae82-275f1e12ec3f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067368 Jul 31 01:20:29.770278 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 537/1073] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:20:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:21:21.304751 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=5 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:21.305711 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=11 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:21.310512 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:21.311134 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:21.319528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:21.320013 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:21.320828 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:21.321294 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:21.353993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:21:21.474601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b15d9f29-76b0-4e5e-9dcf-ac377d2e6c6c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "DELETE /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 204 len: 0 microversion: 2.1 time: 0.171163 Jul 31 01:21:21.475437 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 537/1074] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:21:21 2022] DELETE /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:21:21.482566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=7 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:21.483308 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=46 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:21.488292 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:21.489043 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:21.496965 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:21.497425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:21.498225 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:21.498673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:21.680556 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:21.687696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-bc1bac81-5195-408d-a716-cdb28031b75e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1576 microversion: 2.1 time: 0.206794 Jul 31 01:21:21.688295 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 538/1075] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:21 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1576 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:22.711111 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:22.711973 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:22.720531 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:22.721170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:22.722115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:22.722669 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:22.898748 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:22.907284 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bdecd4c1-bce8-4993-a2da-74a6dc2bf066 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1576 microversion: 2.1 time: 0.201043 Jul 31 01:21:22.907931 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 538/1076] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:22 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1576 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:23.934979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:23.936657 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:23.948912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:23.949486 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:23.950246 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:23.950691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:24.525151 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:24.540561 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e158f002-ff19-45df-937a-798b2e27a8bd tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1576 microversion: 2.1 time: 0.612710 Jul 31 01:21:24.550533 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 539/1077] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:23 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1576 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:24.858926 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-756e516e-a3a0-41fd-9508-d3a390e314c8 req-bd4e99ff-9516-4d3a-890e-06c0b15b2486 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d3c5a74-7a04-4f8b-a068-e68566b7d494", "name": "network-vif-deleted", "tag": "55257163-7d49-4d57-9f0c-9a47106164ab"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:21:24.870884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-756e516e-a3a0-41fd-9508-d3a390e314c8 req-bd4e99ff-9516-4d3a-890e-06c0b15b2486 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:24.871478 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-756e516e-a3a0-41fd-9508-d3a390e314c8 req-bd4e99ff-9516-4d3a-890e-06c0b15b2486 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:24.884667 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-756e516e-a3a0-41fd-9508-d3a390e314c8 req-bd4e99ff-9516-4d3a-890e-06c0b15b2486 service nova] Creating event network-vif-deleted:55257163-7d49-4d57-9f0c-9a47106164ab for instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 on hv-cinder-maste Jul 31 01:21:24.894142 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-756e516e-a3a0-41fd-9508-d3a390e314c8 req-bd4e99ff-9516-4d3a-890e-06c0b15b2486 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038765 Jul 31 01:21:24.896047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 539/1078] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:21:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:21:25.567037 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:25.568572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:25.576709 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:25.577654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:25.578653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:25.580436 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:25.628634 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:25.629181 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:25.629877 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lazy-loading 'fault' on Instance uuid 0d3c5a74-7a04-4f8b-a068-e68566b7d494 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:21:25.676722 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:25.686572 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6b5e7dd8-f918-403d-9181-67032046d4a6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 200 len: 1231 microversion: 2.1 time: 0.124633 Jul 31 01:21:25.687405 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 540/1079] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:25 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 1231 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:26.712672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:26.713583 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:26.721525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:26.722003 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:26.722830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:26.723270 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:26.738534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] HTTP exception thrown: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Request method failure captured: Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 HTTP/1.1 Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dgX8v9xfGsv2W21_rSpypTEJ6GaL-6YHGv4eBG7yVZ6kv6S3ajiAym-Fi_pK5eetm1-mtJgXLrJYGXK0E03hH1XhakO0xu7ellriR5tMzfhjIQ2di1HKOU2xo20fGDF6fBiKgXOxKlsItmAXM7lgTdj5DQScH4Umt_LIKyaBopgXZE Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:21:26.740286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_145c3d8e13bd4e2793bc930d8638eac1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 6a00c618dd284f159d88ad41c0390c5e Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:21:26.742929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '0d3c5a74-7a04-4f8b-a068-e68566b7d494'} Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:21:26.744765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:21:26.750871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:21:26.753166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Returning 404 to user: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:21:26.753166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ca997a6a-c7aa-49a5-b61c-09e7aa4e9fc1 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 404 len: 111 microversion: 2.1 time: 0.035521 Jul 31 01:21:26.753166 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 540/1080] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:26 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:21:26.873003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-485417459", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1012408600", "security_groups": [{"name": "tempest-secgroup-smoke-108034258"}], "block_device_mapping_v2": [{"uuid": "7e83a856-9bae-46b8-a054-f0db55b2f035", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3baf6cd9-bfb5-4744-8ae2-8508343cba8a"}]}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:21:27.010913 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.network.neutron [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] validate_networks() for [('3baf6cd9-bfb5-4744-8ae2-8508343cba8a', None, None, None, None, None)] {{(pid=117942) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:21:27.612999 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Flavor limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:21:27.613557 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Image limits 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:21:27.614087 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Flavor pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:21:27.614549 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Image pref 0:0:0 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:21:27.615010 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.virt.hardware [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117942) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:21:27.625482 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:27.625936 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:27.626767 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:27.627210 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:27.646430 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:21:27.652241 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for user 6a00c618dd284f159d88ad41c0390c5e and project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'instances', 'ram', 'cores'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:21:27.751426 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Going to run 1 instances... {{(pid=117942) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:21:27.753792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='7e83a856-9bae-46b8-a054-f0db55b2f035',volume_size=None,volume_type=None)] {{(pid=117942) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:21:27.918530 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4975d043-231f-41aa-870a-cd7acd8f93c7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.050414 Jul 31 01:21:27.919474 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 541/1081] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:21:26 2022] POST /compute/v2.1/servers => generated 405 bytes in 1052 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 31 01:21:27.933558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-fba5850c-2a42-4823-94db-8e16136573ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:27.934233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-fba5850c-2a42-4823-94db-8e16136573ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:28.011547 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-fba5850c-2a42-4823-94db-8e16136573ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:28.020414 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-fba5850c-2a42-4823-94db-8e16136573ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1094 microversion: 2.1 time: 0.092345 Jul 31 01:21:28.021147 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 541/1082] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:27 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1094 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:29.052446 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:29.053253 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:29.077265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:29.078500 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:29.079912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:29.080538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:29.201826 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:29.211203 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2fd83429-ecd2-4ed3-85a9-3b4ec27f9330 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1200 microversion: 2.1 time: 0.163803 Jul 31 01:21:29.211959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 542/1083] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:29 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1200 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:30.237555 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:30.238942 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:30.250679 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:30.251548 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:30.252990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:30.253865 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:30.487556 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:30.497198 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f39ecd78-0ca1-4fac-bd62-dfc5b88fd765 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.266756 Jul 31 01:21:30.497807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 542/1084] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:30 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:30.955872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-23b9a618-0343-4f35-bab2-d14c6b4dae33 req-b06da487-84fd-4bd9-ab92-94a21eb21848 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1315f433-d198-490d-8ea5-145fa6a3f578", "tag": "2771ba0b-80e4-4264-b5e5-bf766a06f7be"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:21:30.978067 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-23b9a618-0343-4f35-bab2-d14c6b4dae33 req-b06da487-84fd-4bd9-ab92-94a21eb21848 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:30.979049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-23b9a618-0343-4f35-bab2-d14c6b4dae33 req-b06da487-84fd-4bd9-ab92-94a21eb21848 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:31.006574 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-23b9a618-0343-4f35-bab2-d14c6b4dae33 req-b06da487-84fd-4bd9-ab92-94a21eb21848 service nova] Creating event network-changed:2771ba0b-80e4-4264-b5e5-bf766a06f7be for instance 1315f433-d198-490d-8ea5-145fa6a3f578 on hv-cinder-maste Jul 31 01:21:31.012835 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-23b9a618-0343-4f35-bab2-d14c6b4dae33 req-b06da487-84fd-4bd9-ab92-94a21eb21848 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063623 Jul 31 01:21:31.013474 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 543/1085] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:21:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:21:31.532239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:31.533244 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:31.540504 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:31.541279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:31.542092 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:31.542529 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:31.735326 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:31.743346 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ef82d0a2-3cb3-415e-adb8-1fe8d676719c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.216663 Jul 31 01:21:31.743981 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 543/1086] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:31 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:32.766948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:32.767886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:32.777263 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:32.777861 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:32.779059 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:32.779627 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:33.000477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:33.008578 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8c5c8d02-e7bf-4bbb-b43f-57552dc1c2ab tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.247555 Jul 31 01:21:33.009180 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 544/1087] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:32 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:34.029913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:34.030607 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:34.039009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:34.039554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:34.040447 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:34.040938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:34.207220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:34.216158 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-882448ba-4f5b-4d47-941d-0d0b5cb86a3f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.190337 Jul 31 01:21:34.216765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 544/1088] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:34 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:35.238265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:35.238912 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:35.251444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:35.251884 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:35.252683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:35.256102 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:35.418096 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:35.426598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8f43c67b-18e1-4f90-a15a-26face9b357d tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.192929 Jul 31 01:21:35.427269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 545/1089] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:35 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:36.449528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:36.450141 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:36.457559 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:36.458011 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:36.458811 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:36.459550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:36.612541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:36.621499 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-03b40929-9ac6-4d68-8852-e684357bfd40 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.176651 Jul 31 01:21:36.622164 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 545/1090] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:36 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:37.642759 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:37.643406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:37.646411 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:37.653540 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:37.653998 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:37.654831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:37.655279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:37.660299 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=6 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:37.811909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:37.820381 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-6c15433f-e67a-4fb6-817d-45d50a2955d4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.182008 Jul 31 01:21:37.820987 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 546/1091] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:37 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:38.555721 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e227de01-58d2-496a-80a7-c1e5e41ebe60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:38.556597 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e227de01-58d2-496a-80a7-c1e5e41ebe60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:38.561802 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:38.572222 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e227de01-58d2-496a-80a7-c1e5e41ebe60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:38.572960 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e227de01-58d2-496a-80a7-c1e5e41ebe60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:38.574325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e227de01-58d2-496a-80a7-c1e5e41ebe60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:38.575095 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e227de01-58d2-496a-80a7-c1e5e41ebe60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:38.621370 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e227de01-58d2-496a-80a7-c1e5e41ebe60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/ips" status: 200 len: 147 microversion: 2.1 time: 0.072714 Jul 31 01:21:38.622371 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 546/1092] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:21:38 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/ips => generated 147 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:21:38.843026 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:38.843959 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:38.851134 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:38.851905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:38.852476 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:38.852921 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:39.022558 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:39.032071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2b90e208-260a-4117-856c-d6a3384a1897 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.193627 Jul 31 01:21:39.032727 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 547/1093] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:38 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:40.054543 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:40.055814 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:40.063147 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:40.063625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:40.064412 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:40.064853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:40.238779 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:40.247320 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-99dee7d9-dc95-4637-9a48-ae4eee76fb7e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.196899 Jul 31 01:21:40.247970 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 547/1094] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:40 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:40.902312 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-df4c0f2e-ca0f-4c10-a6ed-c2a6acf3c255 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "36f407a2-140f-41e8-a463-dc6d8fc68714"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:21:40.904787 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-df4c0f2e-ca0f-4c10-a6ed-c2a6acf3c255 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:40.911981 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df4c0f2e-ca0f-4c10-a6ed-c2a6acf3c255 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:40.912428 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df4c0f2e-ca0f-4c10-a6ed-c2a6acf3c255 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:40.913281 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df4c0f2e-ca0f-4c10-a6ed-c2a6acf3c255 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:40.913674 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-df4c0f2e-ca0f-4c10-a6ed-c2a6acf3c255 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:41.275534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:41.277816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:41.282820 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-df4c0f2e-ca0f-4c10-a6ed-c2a6acf3c255 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.384342 Jul 31 01:21:41.284092 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 548/1095] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:21:40 2022] POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-volume_attachments => generated 194 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:21:41.291220 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:41.291831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:41.292652 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:41.293106 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:41.543038 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:41.557576 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6a7116d9-7a40-4564-ac8f-040e0bab8e18 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.291026 Jul 31 01:21:41.558710 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 548/1096] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:41 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:42.582329 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:42.583131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:42.590619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:42.591179 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:42.592213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:42.592811 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:42.774804 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-8e865c2b-539d-4f4e-8d90-b40d13a25dde req-fbdf191b-ce3e-43e4-a761-f877b18bdbb2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1315f433-d198-490d-8ea5-145fa6a3f578", "name": "network-vif-plugged", "status": "completed", "tag": "2771ba0b-80e4-4264-b5e5-bf766a06f7be"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:21:42.791275 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-8e865c2b-539d-4f4e-8d90-b40d13a25dde req-fbdf191b-ce3e-43e4-a761-f877b18bdbb2 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:42.792978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-8e865c2b-539d-4f4e-8d90-b40d13a25dde req-fbdf191b-ce3e-43e4-a761-f877b18bdbb2 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:42.795753 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:42.809460 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-8e865c2b-539d-4f4e-8d90-b40d13a25dde req-fbdf191b-ce3e-43e4-a761-f877b18bdbb2 service nova] Creating event network-vif-plugged:2771ba0b-80e4-4264-b5e5-bf766a06f7be for instance 1315f433-d198-490d-8ea5-145fa6a3f578 on hv-cinder-maste Jul 31 01:21:42.810345 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-4c3eb73b-b1ec-4581-9fc9-f6cc36ff1b76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.232722 Jul 31 01:21:42.811646 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 549/1097] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:42 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:42.827703 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-8e865c2b-539d-4f4e-8d90-b40d13a25dde req-fbdf191b-ce3e-43e4-a761-f877b18bdbb2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058136 Jul 31 01:21:42.828994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 549/1098] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:21:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:21:43.845989 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:43.847174 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:43.860263 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:43.861103 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:43.862526 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:43.863692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:44.023719 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:44.033000 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-28dc167d-28a1-487a-8d64-52ed9dfb2cd4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1265 microversion: 2.1 time: 0.195709 Jul 31 01:21:44.033599 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 550/1099] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:43 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1265 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:45.072724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:45.073942 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:45.084131 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:45.084912 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:45.086254 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:45.086924 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:45.241775 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:45.255878 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-bcc0e9e7-b426-41aa-b7b0-d030c9fd0a94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1452 microversion: 2.1 time: 0.191485 Jul 31 01:21:45.256881 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 550/1100] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:45 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1452 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:45.277272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:45.277886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:45.285334 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:45.285851 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:45.286697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:45.287232 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:45.457522 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:45.466355 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-02bae3ec-ac46-47f0-a95f-1283a0936900 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1452 microversion: 2.1 time: 0.193015 Jul 31 01:21:45.466961 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 551/1101] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:45 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1452 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:47.391321 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-735708fc-9b9a-4501-93ef-04a8df5950ea req-a47dca96-5c21-40c4-8b84-e467f268790e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1315f433-d198-490d-8ea5-145fa6a3f578", "tag": "2771ba0b-80e4-4264-b5e5-bf766a06f7be"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:21:47.404122 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-735708fc-9b9a-4501-93ef-04a8df5950ea req-a47dca96-5c21-40c4-8b84-e467f268790e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:47.404789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-735708fc-9b9a-4501-93ef-04a8df5950ea req-a47dca96-5c21-40c4-8b84-e467f268790e service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:47.419503 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-735708fc-9b9a-4501-93ef-04a8df5950ea req-a47dca96-5c21-40c4-8b84-e467f268790e service nova] Creating event network-changed:2771ba0b-80e4-4264-b5e5-bf766a06f7be for instance 1315f433-d198-490d-8ea5-145fa6a3f578 on hv-cinder-maste Jul 31 01:21:47.425774 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-735708fc-9b9a-4501-93ef-04a8df5950ea req-a47dca96-5c21-40c4-8b84-e467f268790e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039704 Jul 31 01:21:47.426642 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 551/1102] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:21:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:21:49.012060 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-54c4b37f-7ff8-455d-84c3-5f6c024a3f40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:21:49.019855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54c4b37f-7ff8-455d-84c3-5f6c024a3f40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:49.020321 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-54c4b37f-7ff8-455d-84c3-5f6c024a3f40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:49.086694 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-54c4b37f-7ff8-455d-84c3-5f6c024a3f40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Going to try to stop instance {{(pid=117942) force_stop /opt/stack/nova/nova/compute/api.py:2709}} Jul 31 01:21:49.137109 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-54c4b37f-7ff8-455d-84c3-5f6c024a3f40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action" status: 202 len: 0 microversion: 2.1 time: 0.128670 Jul 31 01:21:49.137839 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 552/1103] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:21:49 2022] POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action => generated 0 bytes in 131 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 31 01:21:49.153278 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:49.154560 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:49.165115 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:49.165721 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:49.166605 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:49.167826 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:49.580396 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:49.589614 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4c91b89c-6ad8-47bc-bc57-3cb842f7a033 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.445735 Jul 31 01:21:49.590221 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 552/1104] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:49 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:50.612760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:50.613968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:50.621868 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:50.622353 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:50.623158 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:50.623636 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:50.628934 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:50.783050 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:50.793146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d5ab7737-f4dc-47f9-b7a0-4dbad03a17e0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.185610 Jul 31 01:21:50.793917 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 553/1105] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:50 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:51.819702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:51.820656 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:51.824775 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=40,nova_api:UPDATE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:51.835994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:51.836815 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:51.838235 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:51.839619 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:51.848152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:52.038238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:52.047229 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d44db596-f035-4156-8701-caff2e308651 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.235406 Jul 31 01:21:52.047847 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 553/1106] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:51 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:53.070425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:53.071129 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:53.074334 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=48,nova_api:INSERT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:21:53.080220 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:53.080682 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:53.081513 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:53.081965 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:53.242048 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:53.251784 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5be13973-d95e-4a9b-a491-7f2a4ea4badf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.185207 Jul 31 01:21:53.252390 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 554/1107] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:53 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:54.273453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:54.274111 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:54.281858 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:54.282315 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:54.283125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:54.283610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:54.439924 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:54.449728 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-d4e3be91-1ec1-47c8-8e94-82fe62b4b7a4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.180228 Jul 31 01:21:54.450325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 554/1108] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:54 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:55.472152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:55.473855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:55.481435 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:55.481895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:55.482696 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:55.483143 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:55.636642 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:55.645472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a4789124-6f01-4860-842c-d031a563cedf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.178272 Jul 31 01:21:55.646071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 555/1109] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:55 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:21:56.668925 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:56.669992 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:56.678096 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:56.678602 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:56.679518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:56.680023 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:56.856509 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:56.865627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3ac8fe4b-2e90-47a7-9424-29b8d25f7db8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.202400 Jul 31 01:21:56.866305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 555/1110] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:56 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:57.888495 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:57.889233 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:57.904950 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:57.905613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:57.906591 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:57.907140 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:58.071138 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:58.079997 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0d079c77-6b2b-4bd1-877b-f035c2978130 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.195646 Jul 31 01:21:58.080571 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 556/1111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:57 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:21:59.115407 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:21:59.116325 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:21:59.124453 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:59.124908 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:59.125714 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:21:59.126162 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:21:59.284271 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:21:59.293263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e8c0f18d-2a91-4087-b701-359607321e2b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.182245 Jul 31 01:21:59.293873 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 556/1112] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:21:59 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:22:00.316614 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:00.317380 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:00.325153 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:00.325610 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:00.326428 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:00.326874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:00.497815 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:00.507775 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d3467669-f4eb-44c6-a90a-7aaa8a4579ab tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.195769 Jul 31 01:22:00.508738 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 557/1113] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:00 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:22:01.547134 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:01.548306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:01.560793 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:01.561835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:01.563217 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:01.564047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:01.718021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:01.727194 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1ab0fddf-fdfe-4329-87cb-0c3caaf555e1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.188056 Jul 31 01:22:01.727896 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 557/1114] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:01 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:22:02.751403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:02.752113 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:02.760146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:02.761272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:02.762101 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:02.762606 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:02.946714 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:02.961157 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-631e2af2-c6f7-4de4-a5d8-79277b627f2d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.213767 Jul 31 01:22:02.961878 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 558/1115] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:02 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:22:03.997046 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:03.997973 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:04.005476 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:04.006240 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:04.007238 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:04.007870 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:04.195034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:04.211339 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-996663ea-d117-48d3-b483-15879db6882c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1725 microversion: 2.1 time: 0.218665 Jul 31 01:22:04.212429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 558/1116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:03 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1725 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:22:05.234691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:05.235474 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:05.242604 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:05.243187 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:05.244167 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:05.244659 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:05.395992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:05.406330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5f8df84a-9b91-4b9f-a22a-3b02e2e7a07b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1702 microversion: 2.1 time: 0.175724 Jul 31 01:22:05.407131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 559/1117] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:05 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1702 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:22:05.428470 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-f954061f-9907-4a65-a415-0725f9078b98 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:22:05.435940 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f954061f-9907-4a65-a415-0725f9078b98 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:05.436657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-f954061f-9907-4a65-a415-0725f9078b98 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:05.502678 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-f954061f-9907-4a65-a415-0725f9078b98 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Going to try to start instance {{(pid=117943) start /opt/stack/nova/nova/compute/api.py:2732}} Jul 31 01:22:05.552279 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-f954061f-9907-4a65-a415-0725f9078b98 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action" status: 202 len: 0 microversion: 2.1 time: 0.128196 Jul 31 01:22:05.553713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 559/1118] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:22:05 2022] POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action => generated 0 bytes in 130 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:22:05.564487 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:05.565229 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:05.571968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:05.572423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:05.573772 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:05.574213 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:05.794389 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:05.810199 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-ef7cc665-8212-460b-954a-7c2ca3b69d96 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1711 microversion: 2.1 time: 0.250793 Jul 31 01:22:05.811404 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 560/1119] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:05 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1711 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:22:06.839995 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:06.841459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:06.852972 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:06.853758 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:06.855688 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:06.856447 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:07.033335 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:07.049015 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-02459bca-7268-4b40-835f-67bbeb191014 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1711 microversion: 2.1 time: 0.217825 Jul 31 01:22:07.050149 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 560/1120] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:06 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1711 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:22:08.086519 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:22:08.087211 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:22:08.094484 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:08.094948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:08.095789 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:22:08.096252 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:22:08.251673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:22:08.260693 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e30ee643-e834-4f98-bdf0-0627906adb14 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1715 microversion: 2.1 time: 0.178186 Jul 31 01:22:08.261304 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 561/1121] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:22:08 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1715 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:03.693372 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:03.694069 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:03.696699 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=15 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:03.698118 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=51,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:03.705607 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:03.706187 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:03.707049 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:03.707678 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:03.802644 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lazy-loading 'services' on Instance uuid ca751a3a-0157-4e91-b8ca-f6594fca8eb9 {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:23:03.885816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-248b67b3-48fd-4989-a28a-7c7387ca65ca tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-volume_attachments/36f407a2-140f-41e8-a463-dc6d8fc68714" status: 202 len: 0 microversion: 2.1 time: 0.195923 Jul 31 01:23:03.886528 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 561/1122] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:23:03 2022] DELETE /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-volume_attachments/36f407a2-140f-41e8-a463-dc6d8fc68714 => generated 0 bytes in 198 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:23:06.107214 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-aea2565b-9363-4d0c-abba-6fc140615b95 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:23:06.109704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=15 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:06.110359 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=52 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:06.117347 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aea2565b-9363-4d0c-abba-6fc140615b95 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:06.117806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-aea2565b-9363-4d0c-abba-6fc140615b95 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:06.144586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-aea2565b-9363-4d0c-abba-6fc140615b95 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Going to try to stop instance {{(pid=117942) force_stop /opt/stack/nova/nova/compute/api.py:2709}} Jul 31 01:23:06.193251 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-aea2565b-9363-4d0c-abba-6fc140615b95 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action" status: 202 len: 0 microversion: 2.1 time: 0.090412 Jul 31 01:23:06.194139 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 562/1123] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:23:06 2022] POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action => generated 0 bytes in 92 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:23:06.204901 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:06.205620 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:06.212507 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:06.213023 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:06.214280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:06.214280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:06.523930 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:06.532714 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-39274fb6-4210-491c-9a1b-e0e66cbe72dd tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.333138 Jul 31 01:23:06.533322 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 562/1124] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:06 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:07.558338 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:07.559204 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:07.567822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:07.568279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:07.569081 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:07.569529 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:07.788027 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:07.797174 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-fbb25eb9-fc6e-4437-8fea-44de3a871743 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.244929 Jul 31 01:23:07.797775 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 563/1125] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:07 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:08.832143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:08.832754 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:08.839790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:08.840247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:08.841039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:08.841478 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:09.007736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:09.016269 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-eb16a353-41db-45c7-b689-d0928d8e6a60 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.188285 Jul 31 01:23:09.016868 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 563/1126] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:08 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:10.041726 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:10.042869 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:10.054398 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:10.055227 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:10.056677 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:10.057477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:10.213795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:10.228514 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1159422c-74b9-473e-8752-0748f8e672c5 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.193696 Jul 31 01:23:10.229589 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 564/1127] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:10 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:11.264595 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:11.265205 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:11.272383 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:11.272835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:11.273622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:11.274055 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:11.427913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:11.437009 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2201106e-e517-4945-bd84-4278058fce9c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.176368 Jul 31 01:23:11.437895 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 564/1128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:11 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:23:12.458948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:12.459806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:12.466743 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:12.467196 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:12.468016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:12.468464 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:12.613490 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:12.622189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c3c868a4-7cbf-44b6-bc44-81cddebb91a1 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.167492 Jul 31 01:23:12.622859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 565/1129] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:12 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:23:13.645079 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:13.645713 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:13.653950 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:13.654411 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:13.655243 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:13.655724 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:13.811488 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:13.820233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-5a052fea-d9a2-4123-bc4a-99d7c8fd276b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.179080 Jul 31 01:23:13.820831 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 565/1130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:13 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:14.843180 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:14.843824 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:14.851033 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:14.851525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:14.852338 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:14.852786 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:15.002922 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:15.011996 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-f171895b-f166-434b-8895-657ea3c4ec30 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.172751 Jul 31 01:23:15.012627 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 566/1131] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:14 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:23:16.034978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:16.035631 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:16.042523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:16.042993 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:16.043807 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:16.044245 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:16.205763 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:16.214710 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-6fde1b5d-7ae5-41c1-af49-3e441fdce63c tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1679 microversion: 2.1 time: 0.184130 Jul 31 01:23:16.215314 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 566/1132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:16 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1679 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:17.238212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:17.239234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:17.251406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:17.252071 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:17.253584 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:17.254316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:17.421653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:17.430676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b2f715f6-fff7-4bbf-b4f8-97abaea33775 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1656 microversion: 2.1 time: 0.198209 Jul 31 01:23:17.431272 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 567/1133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:17 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1656 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:17.453037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a059a742-bd4f-46eb-a443-ae6682401e6b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:23:17.460267 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a059a742-bd4f-46eb-a443-ae6682401e6b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:17.460810 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a059a742-bd4f-46eb-a443-ae6682401e6b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:17.485915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a059a742-bd4f-46eb-a443-ae6682401e6b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Going to try to start instance {{(pid=117943) start /opt/stack/nova/nova/compute/api.py:2732}} Jul 31 01:23:17.536067 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a059a742-bd4f-46eb-a443-ae6682401e6b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action" status: 202 len: 0 microversion: 2.1 time: 0.086725 Jul 31 01:23:17.536594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 567/1134] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Jul 31 01:23:17 2022] POST /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:23:17.545239 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:17.545875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:17.553496 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:17.554015 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:17.554841 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:17.555290 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:17.757121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:17.769372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-309fbf8a-a220-4383-abea-0ccfb2ff8eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1665 microversion: 2.1 time: 0.228178 Jul 31 01:23:17.770042 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 568/1135] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:17 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1665 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:18.793681 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:18.794525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:18.801777 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:18.802236 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:18.803062 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:18.803554 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:19.028883 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:19.037550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e59e2ebf-0682-4a23-9a45-987b872068a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1665 microversion: 2.1 time: 0.249148 Jul 31 01:23:19.038456 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 568/1136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:18 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1665 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:23:19.267118 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:23:19.267782 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:23:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:23:20.059012 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:20.059643 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:20.067146 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:20.067637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:20.068419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:20.068858 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:20.211296 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:20.219771 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-270c199d-551d-440a-ae7a-a79a8310a789 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1669 microversion: 2.1 time: 0.164598 Jul 31 01:23:20.220542 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 569/1137] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:20 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1669 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:23:26.082338 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:23:26.083027 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Sun Jul 31 01:23:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:23:57.997008 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1605274493", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1012408600", "security_groups": [{"name": "tempest-secgroup-smoke-108034258"}], "block_device_mapping_v2": [{"uuid": "516c7492-d2b5-4518-9965-a606e3d5c372", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "3baf6cd9-bfb5-4744-8ae2-8508343cba8a"}]}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:23:58.008470 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=16 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:58.009170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=55,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:58.146567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] validate_networks() for [('3baf6cd9-bfb5-4744-8ae2-8508343cba8a', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:23:58.694593 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:23:58.695021 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:23:58.695570 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:23:58.696018 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:23:58.696947 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:23:58.706913 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:58.707572 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:58.708476 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:23:58.709006 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:23:58.730053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:23:58.734745 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Getting quotas for user 6a00c618dd284f159d88ad41c0390c5e and project 145c3d8e13bd4e2793bc930d8638eac1. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:23:58.805640 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:23:58.808059 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='516c7492-d2b5-4518-9965-a606e3d5c372',volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:23:58.962438 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b11d3c32-5d45-4a7a-b951-fc5f29957656 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.969867 Jul 31 01:23:58.962994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 569/1138] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:23:57 2022] POST /compute/v2.1/servers => generated 405 bytes in 971 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 31 01:23:58.975670 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-1d796783-c0dc-40b0-9f86-b7c082964937 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:23:58.976568 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-1d796783-c0dc-40b0-9f86-b7c082964937 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:23:58.979005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=16 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:58.979752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:23:59.049925 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-1d796783-c0dc-40b0-9f86-b7c082964937 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:23:59.059003 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-1d796783-c0dc-40b0-9f86-b7c082964937 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1095 microversion: 2.1 time: 0.088094 Jul 31 01:23:59.059680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 570/1139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:23:58 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1095 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:00.082678 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:00.083789 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:00.095864 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:00.096651 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:00.097846 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:00.098550 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:00.201395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:00.216938 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2baa9513-8ddb-42a6-ae9d-78163eba8875 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1201 microversion: 2.1 time: 0.138680 Jul 31 01:24:00.230047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 570/1140] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:00 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1201 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:01.243575 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:01.244949 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:01.258723 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:01.259587 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:01.261409 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:01.262201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:01.569777 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:01.584964 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-09cdd957-0a90-47b7-951b-f973cde48c43 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.349491 Jul 31 01:24:01.586053 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 571/1141] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:01 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:02.095170 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-3fe9e3b6-6f34-40e3-97d9-4cf8bc5d3d89 req-e7fd04ef-f4af-454a-b85d-d93acc5810c7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c11fd85-9310-4844-9033-209c2dd67dc4", "tag": "553efac7-c094-4a42-8598-6251ea29ada5"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:02.116090 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-3fe9e3b6-6f34-40e3-97d9-4cf8bc5d3d89 req-e7fd04ef-f4af-454a-b85d-d93acc5810c7 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:02.117060 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-3fe9e3b6-6f34-40e3-97d9-4cf8bc5d3d89 req-e7fd04ef-f4af-454a-b85d-d93acc5810c7 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:02.138492 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-3fe9e3b6-6f34-40e3-97d9-4cf8bc5d3d89 req-e7fd04ef-f4af-454a-b85d-d93acc5810c7 service nova] Creating event network-changed:553efac7-c094-4a42-8598-6251ea29ada5 for instance 1c11fd85-9310-4844-9033-209c2dd67dc4 on hv-cinder-maste Jul 31 01:24:02.149902 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-3fe9e3b6-6f34-40e3-97d9-4cf8bc5d3d89 req-e7fd04ef-f4af-454a-b85d-d93acc5810c7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059652 Jul 31 01:24:02.151797 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 571/1142] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:24:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:24:02.621279 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:02.622309 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:02.634937 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:02.636052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:02.637444 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:02.638205 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:02.819016 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:02.828351 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-a638b107-bfce-4b56-84df-6b2883d2d7d6 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.215252 Jul 31 01:24:02.829130 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 572/1143] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:02 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:03.851321 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:03.852575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:03.860347 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:03.861077 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:03.862160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:03.862796 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:04.060058 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:04.070097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b5dcde8f-1cd1-4cfd-87c7-6aca7d0033ea tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.223967 Jul 31 01:24:04.070689 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 572/1144] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:03 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:05.099943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:05.100643 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:05.107757 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:05.108212 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:05.108997 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:05.109441 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:05.254327 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:05.262234 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-27d16fbb-f594-4621-8f04-f58fa3551208 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.166464 Jul 31 01:24:05.262831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 573/1145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:05 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:06.283215 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:06.283838 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:06.291718 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:06.292197 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:06.292976 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:06.293409 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:06.449359 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:06.457633 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e8edcf7b-7c91-4c29-b0e7-b23966112398 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.178346 Jul 31 01:24:06.458263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 573/1146] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:06 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:07.479770 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:07.480443 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:07.487501 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:07.487958 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:07.488760 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:07.489204 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:07.639673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:07.648005 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-27714bc7-f3d3-4d99-9971-e6b80fb2d585 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.172689 Jul 31 01:24:07.648600 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 574/1147] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:07 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:08.682677 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:08.683449 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:08.690746 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:08.691198 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:08.692036 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:08.692471 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:08.719473 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:08.742599 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:08.849026 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:08.857991 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c5899d5c-0c99-4f06-a6d2-b4263ea02f22 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.179516 Jul 31 01:24:08.858589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 574/1148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:08 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:09.885294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:09.885909 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:09.888527 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=2 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:09.897824 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:09.898477 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:09.899645 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:09.900282 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:10.080424 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:10.093082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c5551262-b62c-4a1e-bc1d-25c66e3fc382 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.212387 Jul 31 01:24:10.093689 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 575/1149] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:09 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:11.128857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:11.129545 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:11.138112 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:11.138534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:11.139429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:11.140053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:11.293404 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:11.302306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-64d9ca58-316e-4668-ae82-51f140ff3ee7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.177174 Jul 31 01:24:11.302898 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 575/1150] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:11 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:12.325549 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:12.326121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:12.333423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:12.333875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:12.334671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:12.335106 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:12.486712 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:12.494866 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7fe1a820-408f-45e4-a476-80d6a856f8a2 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.173951 Jul 31 01:24:12.495504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 576/1151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:12 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:13.516053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:13.516977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:13.524857 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:13.525315 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:13.526564 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:13.527270 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:13.686385 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:13.700984 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a41483f5-f382-4767-8795-38e2c923f08b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.188555 Jul 31 01:24:13.702054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 576/1152] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:13 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:14.723023 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:14.723730 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:14.731306 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:14.731794 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:14.732611 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:14.733052 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:14.886559 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:14.900080 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-d70f9c71-0728-4cc2-981b-588d2f5a0fc4 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.181078 Jul 31 01:24:14.901035 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 577/1153] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:14 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:15.434232 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-05532525-4863-4fb5-bf03-d692265a8227 req-88e8f335-efaf-4b0d-a7aa-257dddfa2188 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c11fd85-9310-4844-9033-209c2dd67dc4", "name": "network-vif-plugged", "status": "completed", "tag": "553efac7-c094-4a42-8598-6251ea29ada5"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:15.448082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-05532525-4863-4fb5-bf03-d692265a8227 req-88e8f335-efaf-4b0d-a7aa-257dddfa2188 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:15.448622 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-05532525-4863-4fb5-bf03-d692265a8227 req-88e8f335-efaf-4b0d-a7aa-257dddfa2188 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:15.462589 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-05532525-4863-4fb5-bf03-d692265a8227 req-88e8f335-efaf-4b0d-a7aa-257dddfa2188 service nova] Creating event network-vif-plugged:553efac7-c094-4a42-8598-6251ea29ada5 for instance 1c11fd85-9310-4844-9033-209c2dd67dc4 on hv-cinder-maste Jul 31 01:24:15.468353 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-05532525-4863-4fb5-bf03-d692265a8227 req-88e8f335-efaf-4b0d-a7aa-257dddfa2188 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038043 Jul 31 01:24:15.469417 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 577/1154] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:24:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:24:15.936538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:15.937855 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:15.950493 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:15.951306 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:15.952801 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:15.953608 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:16.130643 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:16.141201 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b2f80d6f-a519-4f02-872a-efd63b0aad5f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.211989 Jul 31 01:24:16.141821 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 578/1155] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:15 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:17.165830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:17.166600 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:17.174335 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:17.174830 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:17.175716 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:17.176256 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:17.350487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:17.359107 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cec92325-ea76-4a43-b03e-4084c27cde7f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1266 microversion: 2.1 time: 0.198189 Jul 31 01:24:17.359994 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 578/1156] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:17 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1266 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:18.380849 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:18.381419 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:18.388269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:18.388718 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:18.389495 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:18.389934 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:18.541010 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:18.549172 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-207b865f-6759-483e-9b8a-3fbe361658b7 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1452 microversion: 2.1 time: 0.172094 Jul 31 01:24:18.549779 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 579/1157] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:18 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1452 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:18.594308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:18.594935 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:18.602114 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:18.603134 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:18.603964 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:18.604405 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:18.756197 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:18.764987 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ca40449d-e715-42d5-8969-abdbe0551404 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1452 microversion: 2.1 time: 0.175341 Jul 31 01:24:18.765591 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 579/1158] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:18 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1452 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:21.086699 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-53f472db-13bd-462e-bf36-c88ae256960f req-5c05c0e7-2dd8-44cb-b450-de60435dcd1a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c11fd85-9310-4844-9033-209c2dd67dc4", "tag": "553efac7-c094-4a42-8598-6251ea29ada5"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:21.098184 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-53f472db-13bd-462e-bf36-c88ae256960f req-5c05c0e7-2dd8-44cb-b450-de60435dcd1a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:21.098956 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-53f472db-13bd-462e-bf36-c88ae256960f req-5c05c0e7-2dd8-44cb-b450-de60435dcd1a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:21.114547 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-53f472db-13bd-462e-bf36-c88ae256960f req-5c05c0e7-2dd8-44cb-b450-de60435dcd1a service nova] Creating event network-changed:553efac7-c094-4a42-8598-6251ea29ada5 for instance 1c11fd85-9310-4844-9033-209c2dd67dc4 on hv-cinder-maste Jul 31 01:24:21.123570 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-53f472db-13bd-462e-bf36-c88ae256960f req-5c05c0e7-2dd8-44cb-b450-de60435dcd1a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.146218 Jul 31 01:24:21.124475 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 580/1159] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:24:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:24:25.596425 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ed9f742c-90d5-4ab4-b6f4-efc9940a7a40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:25.600196 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ed9f742c-90d5-4ab4-b6f4-efc9940a7a40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:25.611584 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed9f742c-90d5-4ab4-b6f4-efc9940a7a40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:25.612388 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed9f742c-90d5-4ab4-b6f4-efc9940a7a40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:25.613817 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed9f742c-90d5-4ab4-b6f4-efc9940a7a40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:25.614613 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ed9f742c-90d5-4ab4-b6f4-efc9940a7a40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:25.650515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ed9f742c-90d5-4ab4-b6f4-efc9940a7a40 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.061761 Jul 31 01:24:25.651124 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 580/1160] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:24:25 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9/os-volume_attachments => generated 25 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:24:25.731089 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:25.731780 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:25.738859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:25.739330 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:25.740318 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:25.740646 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:25.771091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:24:25.862215 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0138ad8b-b00f-418a-9786-186883d49dd9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 204 len: 0 microversion: 2.1 time: 0.142109 Jul 31 01:24:25.863149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 581/1161] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:24:25 2022] DELETE /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:24:25.875661 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:25.876770 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:25.889446 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:25.890136 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:25.891927 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:25.892625 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:26.046786 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:26.061264 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-7f858a57-4666-4ac5-92d7-1d352d7d7043 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1675 microversion: 2.1 time: 0.190453 Jul 31 01:24:26.062313 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 581/1162] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:25 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1675 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:27.099069 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:27.099798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:27.106242 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:27.106691 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:27.107505 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:27.107946 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:27.254681 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:27.263758 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-492dc54e-2685-4ffc-bad5-15c1d62cf84d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1675 microversion: 2.1 time: 0.168844 Jul 31 01:24:27.264349 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 582/1163] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:27 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1675 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:28.286790 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:28.287692 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:28.291541 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:28.298670 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:28.299124 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:28.300053 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:28.300487 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:28.304610 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=67 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:28.468286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:28.482935 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c6c4162e-efb3-452f-808b-5a8bb340b5c6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1675 microversion: 2.1 time: 0.200125 Jul 31 01:24:28.483917 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 582/1164] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:28 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1675 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:29.509785 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:29.511138 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:29.515651 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:29.528439 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:29.529235 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:29.530690 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:29.531560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:29.541790 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:29.664286 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:29.672905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-730209e8-d4f2-4339-8943-263733b4357e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 200 len: 1607 microversion: 2.1 time: 0.171220 Jul 31 01:24:29.673403 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 583/1165] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:29 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 1607 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:29.954756 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-a823da79-ddb7-4e2a-8058-7cbcae23b2f8 req-3762d22d-b10e-4d91-8917-04ae41cddf65 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ca751a3a-0157-4e91-b8ca-f6594fca8eb9", "name": "network-vif-deleted", "tag": "2bf3dbb7-a8f6-41f1-92f3-95c49ff50266"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:29.973332 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a823da79-ddb7-4e2a-8058-7cbcae23b2f8 req-3762d22d-b10e-4d91-8917-04ae41cddf65 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:29.974123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a823da79-ddb7-4e2a-8058-7cbcae23b2f8 req-3762d22d-b10e-4d91-8917-04ae41cddf65 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:29.998135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-a823da79-ddb7-4e2a-8058-7cbcae23b2f8 req-3762d22d-b10e-4d91-8917-04ae41cddf65 service nova] Creating event network-vif-deleted:2bf3dbb7-a8f6-41f1-92f3-95c49ff50266 for instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 on hv-cinder-maste Jul 31 01:24:30.006794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-a823da79-ddb7-4e2a-8058-7cbcae23b2f8 req-3762d22d-b10e-4d91-8917-04ae41cddf65 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.178900 Jul 31 01:24:30.007800 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 583/1166] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:24:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:24:30.708804 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:30.709673 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:30.720160 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:30.720979 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:30.722418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:30.723258 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:30.748617 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] HTTP exception thrown: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Request method failure captured: Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 HTTP/1.1 Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5djGPdmhK97MxEy13kOqk9WeTKpwfJkusfvj-HpnKAeHO9KeqkJes00Ee5aWZkUHVQtlkPQqLpnFd9iN8cMFpX5bfS-syb_FIQrc1fUXdg9Oj0CAoUi7QcEfg7ojMbPoiMF9LY15L_YS5AVtmLnaJZVk0rwXyYZWKUGHQywFuioXt6g Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:24:30.751795 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/17c952edcc5440989be309944a4c0b00"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_17c952edcc5440989be309944a4c0b00", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 64fea53260874eae8c13416c97cfa4c7 Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:24:30.754504 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'ca751a3a-0157-4e91-b8ca-f6594fca8eb9'} Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:24:30.755818 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:24:30.758182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:24:30.760943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Returning 404 to user: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:24:30.760943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-0c9368d1-a86a-43ff-8dd0-ba5df93473ed tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 404 len: 111 microversion: 2.1 time: 0.049537 Jul 31 01:24:30.760943 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 584/1167] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:30 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:24:30.767915 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-68a8abbf-5b14-4140-a132-03d7f8d7b453 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '.version_select at 0x7f3e8005eca0>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:30.794051 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-68a8abbf-5b14-4140-a132-03d7f8d7b453 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1558543546" status: 202 len: 0 microversion: 2.1 time: 0.031226 Jul 31 01:24:30.794714 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 584/1168] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Jul 31 01:24:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1558543546 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:24:31.644061 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-36e83380-7dac-4bd7-9c85-7567be27aa33 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'create', calling method: .version_select at 0x7f3e7b5a78b0>, body: {"keypair": {"name": "tempest-keypair-1917060292", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIZAqGBVE/uHNpddmNznNhZHa5WROy5zhBaX+CRsHANLcVwgkDhLA5AgTwgrXCKh+nPkoIHDvoa5fUTTZmMUQPz1YorwfpkmncNeK/atAT/wN9wtCH+3OLvLX+zrXUuE5w=="}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:31.661357 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-36e83380-7dac-4bd7-9c85-7567be27aa33 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for project 17c952edcc5440989be309944a4c0b00. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:24:31.666653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.quota [None req-36e83380-7dac-4bd7-9c85-7567be27aa33 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for user 64fea53260874eae8c13416c97cfa4c7 and project 17c952edcc5440989be309944a4c0b00. Resources: {'key_pairs'} {{(pid=117942) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:24:31.696451 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-36e83380-7dac-4bd7-9c85-7567be27aa33 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.056602 Jul 31 01:24:31.697149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 585/1169] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Jul 31 01:24:31 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:24:33.188240 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-399430975", "imageRef": "33a8368f-2801-48ac-be8e-24b148225061", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "164a7c25-e57c-4866-8228-8acca369f667"}], "security_groups": [{"name": "tempest-securitygroup--330055303"}], "key_name": "tempest-keypair-1917060292", "user_data": "}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:33.303137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.network.neutron [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] validate_networks() for [('164a7c25-e57c-4866-8228-8acca369f667', None, None, None, None, None)] {{(pid=117943) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 31 01:24:33.825542 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Flavor limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 31 01:24:33.826108 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Image limits 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 31 01:24:33.826700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Flavor pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 31 01:24:33.827260 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Image pref 0:0:0 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 31 01:24:33.827815 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.virt.hardware [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=117943) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 31 01:24:33.838120 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:33.838577 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:33.839433 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:33.839884 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:33.861929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for project 17c952edcc5440989be309944a4c0b00. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 31 01:24:33.866522 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.quota [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Getting quotas for user 64fea53260874eae8c13416c97cfa4c7 and project 17c952edcc5440989be309944a4c0b00. Resources: {'instances', 'ram', 'cores'} {{(pid=117943) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 31 01:24:33.879966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Going to run 1 instances... {{(pid=117943) _provision_instances /opt/stack/nova/nova/compute/api.py:1391}} Jul 31 01:24:33.882465 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='33a8368f-2801-48ac-be8e-24b148225061',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=117943) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1809}} Jul 31 01:24:33.949685 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e94a4be4-98d7-47b3-941e-af9b5a72d0db tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.766095 Jul 31 01:24:33.950945 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 585/1170] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Jul 31 01:24:33 2022] POST /compute/v2.1/servers => generated 401 bytes in 769 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 31 01:24:33.971099 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-20789285-b333-41af-8a1d-c2b4c950c084 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:33.972360 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-20789285-b333-41af-8a1d-c2b4c950c084 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:34.056002 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-20789285-b333-41af-8a1d-c2b4c950c084 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:34.062831 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-20789285-b333-41af-8a1d-c2b4c950c084 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1237 microversion: 2.1 time: 0.099850 Jul 31 01:24:34.063971 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 586/1171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:33 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1237 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:35.085432 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:35.086780 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:35.093731 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:35.094179 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:35.094966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:35.095439 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:35.186328 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:35.198306 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ba0cb0c8-1089-48e6-8e1b-d2950c3dc75e tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1289 microversion: 2.1 time: 0.116982 Jul 31 01:24:35.199650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 586/1172] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:35 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1289 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:36.230111 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:36.231720 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:36.244099 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:36.244612 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:36.245563 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:36.246152 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:36.555401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:36.569191 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c351a259-f3ad-4449-860a-d7384a024b17 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.347546 Jul 31 01:24:36.570294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 587/1173] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:36 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:37.022751 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-e2b0b78a-44de-43bf-b327-825ad9d18379 req-f7a1bfec-2692-4ad4-9a9f-de8ec5f57e41 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a3f4b771-1726-47a7-9897-bbd002052ab4", "tag": "f9d86418-8f6a-459d-9f0b-48c8f18b9620"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:37.042861 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-e2b0b78a-44de-43bf-b327-825ad9d18379 req-f7a1bfec-2692-4ad4-9a9f-de8ec5f57e41 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:37.043398 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-e2b0b78a-44de-43bf-b327-825ad9d18379 req-f7a1bfec-2692-4ad4-9a9f-de8ec5f57e41 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:37.057430 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-e2b0b78a-44de-43bf-b327-825ad9d18379 req-f7a1bfec-2692-4ad4-9a9f-de8ec5f57e41 service nova] Creating event network-changed:f9d86418-8f6a-459d-9f0b-48c8f18b9620 for instance a3f4b771-1726-47a7-9897-bbd002052ab4 on hv-cinder-maste Jul 31 01:24:37.064247 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-e2b0b78a-44de-43bf-b327-825ad9d18379 req-f7a1bfec-2692-4ad4-9a9f-de8ec5f57e41 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047933 Jul 31 01:24:37.064875 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 587/1174] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:24:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:24:37.604357 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:37.605230 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:37.611992 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:37.612450 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:37.613267 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:37.613711 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:37.757999 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:37.766150 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cba0a374-d644-4860-9c40-f4a493915147 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.166160 Jul 31 01:24:37.766854 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 588/1175] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:37 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:38.787594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:38.788516 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:38.796351 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:38.796812 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:38.797627 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:38.798073 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:38.952261 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:38.966534 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-a946f4a8-4593-4448-8341-2b809cb74eb8 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.182926 Jul 31 01:24:38.967760 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 588/1176] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:38 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:40.000169 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:40.001312 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:40.009783 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:40.010238 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:40.011032 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:40.011543 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:40.241654 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:40.253586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c27c5cdb-e44a-4fce-9331-f4c38cc0f82b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.257266 Jul 31 01:24:40.254434 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 589/1177] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:39 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:41.277606 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:41.278337 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:41.286358 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:41.286891 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:41.287778 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:41.288266 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:41.483310 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:41.497981 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-1dde5e0b-0e58-420d-a162-4b36a4745c19 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.224013 Jul 31 01:24:41.499211 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 589/1178] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:41 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:42.531933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:42.532636 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:42.539948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:42.540400 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:42.541180 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:42.541616 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:42.684844 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:42.694250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-7317ea70-f8a8-4ffa-b221-3b8d3ea970eb tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.166738 Jul 31 01:24:42.694914 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 590/1179] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:42 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:43.642319 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-0e383e69-4e7b-49ed-a3f9-2830f9f9a045 req-8dafe585-3544-467d-9b8b-a562800bd641 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a3f4b771-1726-47a7-9897-bbd002052ab4", "name": "network-vif-plugged", "status": "completed", "tag": "f9d86418-8f6a-459d-9f0b-48c8f18b9620"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:43.658879 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-0e383e69-4e7b-49ed-a3f9-2830f9f9a045 req-8dafe585-3544-467d-9b8b-a562800bd641 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:43.659708 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-0e383e69-4e7b-49ed-a3f9-2830f9f9a045 req-8dafe585-3544-467d-9b8b-a562800bd641 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:43.681743 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-0e383e69-4e7b-49ed-a3f9-2830f9f9a045 req-8dafe585-3544-467d-9b8b-a562800bd641 service nova] Creating event network-vif-plugged:f9d86418-8f6a-459d-9f0b-48c8f18b9620 for instance a3f4b771-1726-47a7-9897-bbd002052ab4 on hv-cinder-maste Jul 31 01:24:43.691147 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-0e383e69-4e7b-49ed-a3f9-2830f9f9a045 req-8dafe585-3544-467d-9b8b-a562800bd641 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.136999 Jul 31 01:24:43.692242 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 590/1180] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:24:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:24:43.715703 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:43.716564 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:43.723225 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:43.723724 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:43.724550 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:43.724977 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:43.916566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:43.924676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-550897a4-c22a-43b0-a516-9e27fe409834 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.212886 Jul 31 01:24:43.925284 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 591/1181] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:43 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:44.943301 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=2 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:44.944156 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell0:SELECT=6 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:44.949366 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:44.950078 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:44.957544 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:44.957996 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:44.958792 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:44.959233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:45.123082 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:45.131084 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-c6aac761-1695-42ae-abc3-ce993a85ed59 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1362 microversion: 2.1 time: 0.188652 Jul 31 01:24:45.131711 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 591/1182] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:44 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1362 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:24:46.152956 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:46.153567 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:46.155451 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell0:SELECT=6 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:24:46.162149 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:46.162598 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:46.163433 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:46.163873 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:46.310102 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:24:46.318250 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cfb6d8b0-0e25-4c86-b6eb-a8b4e50a42d7 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1548 microversion: 2.1 time: 0.169639 Jul 31 01:24:46.318857 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 592/1183] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:24:46 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1548 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:24:46.344386 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-3efbb730-6fcd-4e48-aacc-f581d94776f6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:24:46.345163 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-3efbb730-6fcd-4e48-aacc-f581d94776f6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:24:46.352852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3efbb730-6fcd-4e48-aacc-f581d94776f6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:46.353308 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3efbb730-6fcd-4e48-aacc-f581d94776f6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:46.354108 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3efbb730-6fcd-4e48-aacc-f581d94776f6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:46.354592 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-3efbb730-6fcd-4e48-aacc-f581d94776f6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:46.444382 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-3efbb730-6fcd-4e48-aacc-f581d94776f6 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.104041 Jul 31 01:24:46.445184 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 592/1184] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Jul 31 01:24:46 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-interface => generated 284 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:24:47.762329 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-cd01500d-6c1c-4082-b8f3-f149ee0162b4 req-bb7878db-3486-4735-9d78-6730988524dd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a3f4b771-1726-47a7-9897-bbd002052ab4", "tag": "f9d86418-8f6a-459d-9f0b-48c8f18b9620"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:24:47.780599 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-cd01500d-6c1c-4082-b8f3-f149ee0162b4 req-bb7878db-3486-4735-9d78-6730988524dd service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:24:47.781418 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-cd01500d-6c1c-4082-b8f3-f149ee0162b4 req-bb7878db-3486-4735-9d78-6730988524dd service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:24:47.803845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-cd01500d-6c1c-4082-b8f3-f149ee0162b4 req-bb7878db-3486-4735-9d78-6730988524dd service nova] Creating event network-changed:f9d86418-8f6a-459d-9f0b-48c8f18b9620 for instance a3f4b771-1726-47a7-9897-bbd002052ab4 on hv-cinder-maste Jul 31 01:24:47.812182 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-cd01500d-6c1c-4082-b8f3-f149ee0162b4 req-bb7878db-3486-4735-9d78-6730988524dd service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056451 Jul 31 01:24:47.812814 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 593/1185] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:24:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:03.320227 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=35,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:03.321159 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=41 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:03.389910 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-320a4429-a304-4c3d-880e-8202ab67916d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:03.390562 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-320a4429-a304-4c3d-880e-8202ab67916d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:03.398750 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-320a4429-a304-4c3d-880e-8202ab67916d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:03.399212 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-320a4429-a304-4c3d-880e-8202ab67916d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:03.400137 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-320a4429-a304-4c3d-880e-8202ab67916d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:03.400578 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-320a4429-a304-4c3d-880e-8202ab67916d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:03.434145 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-320a4429-a304-4c3d-880e-8202ab67916d tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/ips" status: 200 len: 145 microversion: 2.1 time: 0.117013 Jul 31 01:26:03.434725 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 593/1186] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Jul 31 01:26:03 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/ips => generated 145 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:04.843898 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-64648f4d-d0dc-46e6-ab51-cb2482c55c62 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d981e825-20f1-4e74-88db-eb89dd3e0f70"}} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:26:04.847296 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-64648f4d-d0dc-46e6-ab51-cb2482c55c62 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:04.850423 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=32,nova_api:INSERT=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:04.851613 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=45 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:04.862494 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-64648f4d-d0dc-46e6-ab51-cb2482c55c62 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:04.863413 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-64648f4d-d0dc-46e6-ab51-cb2482c55c62 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:04.864377 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-64648f4d-d0dc-46e6-ab51-cb2482c55c62 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:04.865017 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-64648f4d-d0dc-46e6-ab51-cb2482c55c62 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:05.232874 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-64648f4d-d0dc-46e6-ab51-cb2482c55c62 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.395318 Jul 31 01:26:05.233590 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 594/1187] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:26:04 2022] POST /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments => generated 194 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:26:12.097215 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:12.098951 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:12.109922 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-d7ad0ee4-f52c-433e-a23a-e915e5a97cc0 req-a1732c19-0feb-4bdf-af9e-a5470a6b6744 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c11fd85-9310-4844-9033-209c2dd67dc4", "tag": "553efac7-c094-4a42-8598-6251ea29ada5"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:26:12.112328 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:12.113329 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:12.114867 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:12.115709 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:12.145425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-d7ad0ee4-f52c-433e-a23a-e915e5a97cc0 req-a1732c19-0feb-4bdf-af9e-a5470a6b6744 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:12.145425 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-d7ad0ee4-f52c-433e-a23a-e915e5a97cc0 req-a1732c19-0feb-4bdf-af9e-a5470a6b6744 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:12.162391 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:26:12.178154 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-d7ad0ee4-f52c-433e-a23a-e915e5a97cc0 req-a1732c19-0feb-4bdf-af9e-a5470a6b6744 service nova] Creating event network-changed:553efac7-c094-4a42-8598-6251ea29ada5 for instance 1c11fd85-9310-4844-9033-209c2dd67dc4 on hv-cinder-maste Jul 31 01:26:12.190462 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-d7ad0ee4-f52c-433e-a23a-e915e5a97cc0 req-a1732c19-0feb-4bdf-af9e-a5470a6b6744 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085958 Jul 31 01:26:12.192706 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 595/1188] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:26:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:26:12.302280 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-879df55c-4d4d-4f1c-a0f3-4f5993be5d94 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 204 len: 0 microversion: 2.1 time: 0.211600 Jul 31 01:26:12.303935 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 594/1189] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:26:12 2022] DELETE /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:26:12.315680 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:12.316508 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:12.323725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:12.324445 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:12.325619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:12.325935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:12.527528 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:12.535784 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-bc93c8d8-8da5-4919-936a-db45ccd450fc tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1577 microversion: 2.1 time: 0.225058 Jul 31 01:26:12.536698 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 596/1190] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:12 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1577 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:26:13.008697 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-ad63b151-4312-40d1-bc13-3af43cf3efc9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:13.011558 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-ad63b151-4312-40d1-bc13-3af43cf3efc9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:13.019213 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ad63b151-4312-40d1-bc13-3af43cf3efc9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:13.019832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ad63b151-4312-40d1-bc13-3af43cf3efc9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:13.020837 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ad63b151-4312-40d1-bc13-3af43cf3efc9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:13.021563 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-ad63b151-4312-40d1-bc13-3af43cf3efc9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:13.066429 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-ad63b151-4312-40d1-bc13-3af43cf3efc9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064255 Jul 31 01:26:13.067829 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 595/1191] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:26:13 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:13.079676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-81bef8f0-54a6-4aaf-b6f4-537271cc2d82 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:13.080317 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-81bef8f0-54a6-4aaf-b6f4-537271cc2d82 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:13.090019 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-81bef8f0-54a6-4aaf-b6f4-537271cc2d82 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:13.090570 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-81bef8f0-54a6-4aaf-b6f4-537271cc2d82 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:13.091755 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-81bef8f0-54a6-4aaf-b6f4-537271cc2d82 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:13.092806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-81bef8f0-54a6-4aaf-b6f4-537271cc2d82 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:13.143392 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-81bef8f0-54a6-4aaf-b6f4-537271cc2d82 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/d981e825-20f1-4e74-88db-eb89dd3e0f70" status: 200 len: 194 microversion: 2.1 time: 0.068107 Jul 31 01:26:13.144453 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 597/1192] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Jul 31 01:26:13 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/d981e825-20f1-4e74-88db-eb89dd3e0f70 => generated 194 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:13.578305 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:13.580300 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:13.602578 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:13.604365 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:13.607135 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:13.608390 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:14.150312 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:14.160294 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-933e4d90-6706-4f88-996d-dafc94c1a370 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1577 microversion: 2.1 time: 0.601239 Jul 31 01:26:14.161075 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 596/1193] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:13 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1577 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:26:14.709161 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-85168723-40dc-403f-b38d-b408b3c459d6 req-6c48a3f5-9a61-4059-8b2b-8f3b170fe790 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c11fd85-9310-4844-9033-209c2dd67dc4", "name": "network-vif-deleted", "tag": "553efac7-c094-4a42-8598-6251ea29ada5"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:26:14.732683 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-85168723-40dc-403f-b38d-b408b3c459d6 req-6c48a3f5-9a61-4059-8b2b-8f3b170fe790 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:14.733904 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-85168723-40dc-403f-b38d-b408b3c459d6 req-6c48a3f5-9a61-4059-8b2b-8f3b170fe790 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:14.743852 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-4aad7fa4-28bf-4956-bb5d-6dae104a4860 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "03eeba2a-65af-44dd-a649-8b7eac64abe3"}} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:26:14.746464 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-4aad7fa4-28bf-4956-bb5d-6dae104a4860 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:14.755125 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4aad7fa4-28bf-4956-bb5d-6dae104a4860 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:14.756548 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4aad7fa4-28bf-4956-bb5d-6dae104a4860 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:14.756548 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4aad7fa4-28bf-4956-bb5d-6dae104a4860 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:14.756952 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-4aad7fa4-28bf-4956-bb5d-6dae104a4860 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:14.765452 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-85168723-40dc-403f-b38d-b408b3c459d6 req-6c48a3f5-9a61-4059-8b2b-8f3b170fe790 service nova] Creating event network-vif-deleted:553efac7-c094-4a42-8598-6251ea29ada5 for instance 1c11fd85-9310-4844-9033-209c2dd67dc4 on hv-cinder-maste Jul 31 01:26:14.779294 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-85168723-40dc-403f-b38d-b408b3c459d6 req-6c48a3f5-9a61-4059-8b2b-8f3b170fe790 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077891 Jul 31 01:26:14.781162 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 598/1194] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:26:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:15.088816 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-4aad7fa4-28bf-4956-bb5d-6dae104a4860 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "POST /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.349166 Jul 31 01:26:15.089890 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 597/1195] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Jul 31 01:26:14 2022] POST /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments => generated 194 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:15.190198 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:15.191524 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:15.207630 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:15.208697 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:15.210017 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:15.210744 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:15.268730 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:15.269242 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:15.270258 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lazy-loading 'fault' on Instance uuid 1c11fd85-9310-4844-9033-209c2dd67dc4 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:26:15.316086 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:15.325237 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-c4000cd5-e934-43dc-a25e-31cef396853c tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 200 len: 1231 microversion: 2.1 time: 0.140955 Jul 31 01:26:15.325852 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 599/1196] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:15 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 1231 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:26:16.347905 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:16.348566 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1c11fd85-9310-4844-9033-209c2dd67dc4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:16.355764 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:16.356224 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:16.357039 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:16.357488 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:16.372929 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] HTTP exception thrown: Instance 1c11fd85-9310-4844-9033-209c2dd67dc4 could not be found. Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Request method failure captured: Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 HTTP/1.1 Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dgX8v9xfGsv2W21_rSpypTEJ6GaL-6YHGv4eBG7yVZ6kv6S3ajiAym-Fi_pK5eetm1-mtJgXLrJYGXK0E03hH1XhakO0xu7ellriR5tMzfhjIQ2di1HKOU2xo20fGDF6fBiKgXOxKlsItmAXM7lgTdj5DQScH4Umt_LIKyaBopgXZE Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:26:16.374647 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_145c3d8e13bd4e2793bc930d8638eac1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 6a00c618dd284f159d88ad41c0390c5e Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 1c11fd85-9310-4844-9033-209c2dd67dc4 could not be found. Jul 31 01:26:16.377176 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '1c11fd85-9310-4844-9033-209c2dd67dc4'} Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c11fd85-9310-4844-9033-209c2dd67dc4 could not be found. Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:16.378515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c11fd85-9310-4844-9033-209c2dd67dc4 could not be found. Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c11fd85-9310-4844-9033-209c2dd67dc4 could not be found. Jul 31 01:26:16.380955 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:16.383772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Returning 404 to user: Instance 1c11fd85-9310-4844-9033-209c2dd67dc4 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:16.383772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-16770c71-143b-4a74-9feb-caac924d5f01 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4" status: 404 len: 111 microversion: 2.1 time: 0.033277 Jul 31 01:26:16.383772 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 598/1197] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:16 2022] GET /compute/v2.1/servers/1c11fd85-9310-4844-9033-209c2dd67dc4 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:26:20.206845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:20.208040 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:20.208455 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-a067bc45-6a78-4754-96bd-856d665f78ee req-7c4950f4-8cad-405c-baed-7dbca26d7774 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1315f433-d198-490d-8ea5-145fa6a3f578", "tag": "2771ba0b-80e4-4264-b5e5-bf766a06f7be"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:26:20.218792 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:20.219311 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a067bc45-6a78-4754-96bd-856d665f78ee req-7c4950f4-8cad-405c-baed-7dbca26d7774 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:20.219729 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:20.220023 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-a067bc45-6a78-4754-96bd-856d665f78ee req-7c4950f4-8cad-405c-baed-7dbca26d7774 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:20.220908 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:20.221538 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:20.233403 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-a067bc45-6a78-4754-96bd-856d665f78ee req-7c4950f4-8cad-405c-baed-7dbca26d7774 service nova] Creating event network-changed:2771ba0b-80e4-4264-b5e5-bf766a06f7be for instance 1315f433-d198-490d-8ea5-145fa6a3f578 on hv-cinder-maste Jul 31 01:26:20.239152 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-a067bc45-6a78-4754-96bd-856d665f78ee req-7c4950f4-8cad-405c-baed-7dbca26d7774 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035074 Jul 31 01:26:20.240317 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 599/1198] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:26:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:20.259604 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Going to try to terminate instance {{(pid=117942) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:26:20.385885 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-beb18a6d-4c15-4416-b217-7519bab65c9f tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 204 len: 0 microversion: 2.1 time: 0.185784 Jul 31 01:26:20.387506 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 600/1199] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:26:20 2022] DELETE /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:26:20.402034 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:20.403871 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:20.418966 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:20.420978 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:20.423508 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:20.424906 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:20.658012 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:20.667701 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-b46a8f88-c78f-4abe-aa52-f95eea81bf6a tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1577 microversion: 2.1 time: 0.275086 Jul 31 01:26:20.683499 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 600/1200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:20 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1577 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:26:21.696560 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:21.697315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:21.704619 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:21.705091 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:21.705905 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:21.706349 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:21.863563 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:21.872031 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-cfcfc084-9d15-454e-82a2-f48b8a68c53b tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1577 microversion: 2.1 time: 0.180362 Jul 31 01:26:21.872638 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 601/1201] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:21 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1577 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:26:22.894552 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:22.895447 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:22.903133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:22.903686 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:22.904524 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:22.905005 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:23.202575 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:23.236385 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8b48840e-2a17-4f7b-949d-ffdefdd14121 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1577 microversion: 2.1 time: 0.345388 Jul 31 01:26:23.266531 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 601/1202] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:22 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1577 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:26:23.646164 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [req-d52032ea-c9fc-4df2-8e77-daaf42ee93c4 req-c891f747-a950-451f-ba2f-de175b4dba52 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1315f433-d198-490d-8ea5-145fa6a3f578", "name": "network-vif-deleted", "tag": "2771ba0b-80e4-4264-b5e5-bf766a06f7be"}]} {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:26:23.667588 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-d52032ea-c9fc-4df2-8e77-daaf42ee93c4 req-c891f747-a950-451f-ba2f-de175b4dba52 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:23.668372 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [req-d52032ea-c9fc-4df2-8e77-daaf42ee93c4 req-c891f747-a950-451f-ba2f-de175b4dba52 service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:23.696269 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.compute.server_external_events [req-d52032ea-c9fc-4df2-8e77-daaf42ee93c4 req-c891f747-a950-451f-ba2f-de175b4dba52 service nova] Creating event network-vif-deleted:2771ba0b-80e4-4264-b5e5-bf766a06f7be for instance 1315f433-d198-490d-8ea5-145fa6a3f578 on hv-cinder-maste Jul 31 01:26:23.706111 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [req-d52032ea-c9fc-4df2-8e77-daaf42ee93c4 req-c891f747-a950-451f-ba2f-de175b4dba52 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067161 Jul 31 01:26:23.707189 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 602/1203] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:26:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:23.889168 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-e475c7ee-3db9-4a48-beec-02e3a84988cf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:23.893286 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-e475c7ee-3db9-4a48-beec-02e3a84988cf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:23.909518 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e475c7ee-3db9-4a48-beec-02e3a84988cf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:23.910350 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e475c7ee-3db9-4a48-beec-02e3a84988cf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:23.911690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e475c7ee-3db9-4a48-beec-02e3a84988cf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:23.912791 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-e475c7ee-3db9-4a48-beec-02e3a84988cf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:23.979798 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-e475c7ee-3db9-4a48-beec-02e3a84988cf tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.098014 Jul 31 01:26:23.981290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 602/1204] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:26:23 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments => generated 371 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:24.131935 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-608e4428-3fc5-4626-855c-169d7f2479a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:24.133580 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-608e4428-3fc5-4626-855c-169d7f2479a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:24.145316 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608e4428-3fc5-4626-855c-169d7f2479a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:24.145886 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608e4428-3fc5-4626-855c-169d7f2479a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:24.146842 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608e4428-3fc5-4626-855c-169d7f2479a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:24.147326 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-608e4428-3fc5-4626-855c-169d7f2479a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:24.207406 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-608e4428-3fc5-4626-855c-169d7f2479a9 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/d981e825-20f1-4e74-88db-eb89dd3e0f70" status: 200 len: 194 microversion: 2.1 time: 0.081040 Jul 31 01:26:24.208405 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 603/1205] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Jul 31 01:26:24 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/d981e825-20f1-4e74-88db-eb89dd3e0f70 => generated 194 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:26:24.221657 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:24.222532 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:24.230054 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:24.230567 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:24.231515 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:24.231990 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:24.274256 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:24.274983 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:24.283635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:24.283635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:24.283635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:24.283635 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:24.338798 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:24.339641 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:24.340469 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lazy-loading 'fault' on Instance uuid 1315f433-d198-490d-8ea5-145fa6a3f578 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:26:24.340698 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.objects.instance [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lazy-loading 'services' on Instance uuid a3f4b771-1726-47a7-9897-bbd002052ab4 {{(pid=117943) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:26:24.441357 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a00c618dd284f159d88ad41c0390c5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145c3d8e13bd4e2793bc930d8638eac1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:24.452907 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-688345fe-cf85-4589-8158-db1843d3bf93 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 200 len: 1230 microversion: 2.1 time: 0.182729 Jul 31 01:26:24.453509 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 604/1206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:24 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 1230 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 31 01:26:24.472395 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-36e4e914-0355-4128-bfa0-9a2a0b37ec1b tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/d981e825-20f1-4e74-88db-eb89dd3e0f70" status: 202 len: 0 microversion: 2.1 time: 0.254546 Jul 31 01:26:24.474413 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 603/1207] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:26:24 2022] DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/d981e825-20f1-4e74-88db-eb89dd3e0f70 => generated 0 bytes in 258 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:26:25.480321 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:25.481207 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 1315f433-d198-490d-8ea5-145fa6a3f578] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:25.488899 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:25.489412 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:25.490265 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:25.491344 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:25.506888 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] HTTP exception thrown: Instance 1315f433-d198-490d-8ea5-145fa6a3f578 could not be found. Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Request method failure captured: Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 HTTP/1.1 Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dgX8v9xfGsv2W21_rSpypTEJ6GaL-6YHGv4eBG7yVZ6kv6S3ajiAym-Fi_pK5eetm1-mtJgXLrJYGXK0E03hH1XhakO0xu7ellriR5tMzfhjIQ2di1HKOU2xo20fGDF6fBiKgXOxKlsItmAXM7lgTdj5DQScH4Umt_LIKyaBopgXZE Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:26:25.508637 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_145c3d8e13bd4e2793bc930d8638eac1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 6a00c618dd284f159d88ad41c0390c5e Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 1315f433-d198-490d-8ea5-145fa6a3f578 could not be found. Jul 31 01:26:25.511068 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '1315f433-d198-490d-8ea5-145fa6a3f578'} Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1315f433-d198-490d-8ea5-145fa6a3f578 could not be found. Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:25.512525 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1315f433-d198-490d-8ea5-145fa6a3f578 could not be found. Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1315f433-d198-490d-8ea5-145fa6a3f578 could not be found. Jul 31 01:26:25.514952 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:25.517018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Returning 404 to user: Instance 1315f433-d198-490d-8ea5-145fa6a3f578 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:25.517018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-8611b049-39c6-4da2-be4e-b4472fc0a220 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578" status: 404 len: 111 microversion: 2.1 time: 0.034056 Jul 31 01:26:25.517018 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 605/1208] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:25 2022] GET /compute/v2.1/servers/1315f433-d198-490d-8ea5-145fa6a3f578 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:26:25.690143 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-2ac3c27c-888c-438d-9ef1-3334eb8181b4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:25.691394 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-2ac3c27c-888c-438d-9ef1-3334eb8181b4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:25.700309 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2ac3c27c-888c-438d-9ef1-3334eb8181b4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:25.700762 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2ac3c27c-888c-438d-9ef1-3334eb8181b4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:25.701809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2ac3c27c-888c-438d-9ef1-3334eb8181b4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:25.702565 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-2ac3c27c-888c-438d-9ef1-3334eb8181b4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:25.745089 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-2ac3c27c-888c-438d-9ef1-3334eb8181b4 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/03eeba2a-65af-44dd-a649-8b7eac64abe3" status: 200 len: 194 microversion: 2.1 time: 0.061710 Jul 31 01:26:25.746037 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 604/1209] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Jul 31 01:26:25 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/03eeba2a-65af-44dd-a649-8b7eac64abe3 => generated 194 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 31 01:26:25.758083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:25.758593 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:25.766367 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:25.767225 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:25.768479 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:25.769497 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:25.868056 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lazy-loading 'services' on Instance uuid a3f4b771-1726-47a7-9897-bbd002052ab4 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:26:25.985219 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-18eed244-a0db-4c1a-88d4-97c6ec440f1f tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/03eeba2a-65af-44dd-a649-8b7eac64abe3" status: 202 len: 0 microversion: 2.1 time: 0.231699 Jul 31 01:26:25.986274 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 606/1210] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Jul 31 01:26:25 2022] DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments/03eeba2a-65af-44dd-a649-8b7eac64abe3 => generated 0 bytes in 234 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:26:26.238461 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:26.239119 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:26.250977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:26.251458 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:26.252241 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:26.252702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:26.267961 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] HTTP exception thrown: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Request method failure captured: Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: DELETE /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 HTTP/1.1 Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5dgX8v9xfGsv2W21_rSpypTEJ6GaL-6YHGv4eBG7yVZ6kv6S3ajiAym-Fi_pK5eetm1-mtJgXLrJYGXK0E03hH1XhakO0xu7ellriR5tMzfhjIQ2di1HKOU2xo20fGDF6fBiKgXOxKlsItmAXM7lgTdj5DQScH4Umt_LIKyaBopgXZE Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:26:26.269672 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_145c3d8e13bd4e2793bc930d8638eac1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 6a00c618dd284f159d88ad41c0390c5e Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.272289 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': '0d3c5a74-7a04-4f8b-a068-e68566b7d494'} Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:26.273824 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.276160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.278765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Returning 404 to user: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:26.278765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-285dca15-b156-4c95-875d-301d0ed1a712 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "DELETE /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 404 len: 111 microversion: 2.1 time: 0.043029 Jul 31 01:26:26.278765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 605/1211] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:26:26 2022] DELETE /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:26:26.287947 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:26.288653 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] [instance: 0d3c5a74-7a04-4f8b-a068-e68566b7d494] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:26.295678 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:26.296178 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:26.296984 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:26.297435 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:26.315283 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] HTTP exception thrown: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Request method failure captured: Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 HTTP/1.1 Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5dgX8v9xfGsv2W21_rSpypTEJ6GaL-6YHGv4eBG7yVZ6kv6S3ajiAym-Fi_pK5eetm1-mtJgXLrJYGXK0E03hH1XhakO0xu7ellriR5tMzfhjIQ2di1HKOU2xo20fGDF6fBiKgXOxKlsItmAXM7lgTdj5DQScH4Umt_LIKyaBopgXZE Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:26:26.317006 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/145c3d8e13bd4e2793bc930d8638eac1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_145c3d8e13bd4e2793bc930d8638eac1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 145c3d8e13bd4e2793bc930d8638eac1 Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-TestVolumeBootPattern-1704098064 Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 6a00c618dd284f159d88ad41c0390c5e Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-TestVolumeBootPattern-1704098064-project Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.319054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': '0d3c5a74-7a04-4f8b-a068-e68566b7d494'} Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:26.320083 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. Jul 31 01:26:26.321516 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:26.322671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Returning 404 to user: Instance 0d3c5a74-7a04-4f8b-a068-e68566b7d494 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:26.322671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-e1b1e061-c88d-42a1-b873-e46717f31a76 tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494" status: 404 len: 111 microversion: 2.1 time: 0.035342 Jul 31 01:26:26.322671 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 607/1212] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:26 2022] GET /compute/v2.1/servers/0d3c5a74-7a04-4f8b-a068-e68566b7d494 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:26:27.198526 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-152f9ac1-d175-4f7c-9962-df440052fe87 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:27.200858 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-152f9ac1-d175-4f7c-9962-df440052fe87 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:27.207887 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-152f9ac1-d175-4f7c-9962-df440052fe87 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.208339 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-152f9ac1-d175-4f7c-9962-df440052fe87 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.209687 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-152f9ac1-d175-4f7c-9962-df440052fe87 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.210133 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-152f9ac1-d175-4f7c-9962-df440052fe87 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.247047 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-152f9ac1-d175-4f7c-9962-df440052fe87 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052546 Jul 31 01:26:27.247783 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 606/1213] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:26:27 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:26:27.384414 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-87f9a93b-3322-4a00-b1da-0669b65791fe tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:27.387862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-87f9a93b-3322-4a00-b1da-0669b65791fe tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:27.394889 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87f9a93b-3322-4a00-b1da-0669b65791fe tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.395342 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87f9a93b-3322-4a00-b1da-0669b65791fe tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.396144 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87f9a93b-3322-4a00-b1da-0669b65791fe tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.397166 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-87f9a93b-3322-4a00-b1da-0669b65791fe tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.435296 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-87f9a93b-3322-4a00-b1da-0669b65791fe tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.057656 Jul 31 01:26:27.440752 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 608/1214] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Jul 31 01:26:27 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4/os-volume_attachments => generated 25 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 31 01:26:27.523293 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:27.524233 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:27.535835 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.536296 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.537150 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.538110 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.569611 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Going to try to terminate instance {{(pid=117943) delete /opt/stack/nova/nova/compute/api.py:2666}} Jul 31 01:26:27.587290 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:27.680930 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0f14c696-0c23-4e86-b124-0e444f5f8b79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 204 len: 0 microversion: 2.1 time: 0.163525 Jul 31 01:26:27.682525 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 607/1215] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:26:27 2022] DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 31 01:26:27.696105 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:27.697141 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:27.708127 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.708974 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.710948 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:27.712082 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:27.750054 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:27.760597 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-0d7903c8-ddf7-4bf1-9705-80ecad2a406e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] Calling method '.version_select at 0x7f3e80116c10>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:27.801268 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-0d7903c8-ddf7-4bf1-9705-80ecad2a406e tempest-TestVolumeBootPattern-1704098064 tempest-TestVolumeBootPattern-1704098064-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1012408600" status: 202 len: 0 microversion: 2.1 time: 0.050436 Jul 31 01:26:27.807234 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 608/1216] 10.222.0.42 () {58 vars in 1263 bytes} [Sun Jul 31 01:26:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1012408600 => generated 0 bytes in 58 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 31 01:26:27.918371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:27.927845 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-b6c19dc7-118b-412b-a16c-0fafc6437e1a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1672 microversion: 2.1 time: 0.240099 Jul 31 01:26:27.928832 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 609/1217] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:27 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1672 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:26:28.960277 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:28.961629 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:28.977773 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:28.978587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:28.980450 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:28.981227 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:29.222671 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.policy [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117943) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:29.234160 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cd9eef91-38bc-48b6-8be9-6fb553e9a18a tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1672 microversion: 2.1 time: 0.283108 Jul 31 01:26:29.234765 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 609/1218] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:28 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1672 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:26:30.267166 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:30.269293 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:30.288030 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:30.290017 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:30.294323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:30.295875 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:30.631749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:30.642586 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-3a1aaa76-4d5c-4d0f-beac-18274787dbc0 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1672 microversion: 2.1 time: 0.387227 Jul 31 01:26:30.646605 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 610/1219] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:30 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1672 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:26:31.261335 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [req-593539d9-4295-44c8-9a8a-b488554fb809 req-292fa595-9b8c-45d2-ba27-a3368482207a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a3f4b771-1726-47a7-9897-bbd002052ab4", "name": "network-vif-deleted", "tag": "f9d86418-8f6a-459d-9f0b-48c8f18b9620"}]} {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 31 01:26:31.311808 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-593539d9-4295-44c8-9a8a-b488554fb809 req-292fa595-9b8c-45d2-ba27-a3368482207a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:31.312754 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [req-593539d9-4295-44c8-9a8a-b488554fb809 req-292fa595-9b8c-45d2-ba27-a3368482207a service nova] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:31.352123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.compute.server_external_events [req-593539d9-4295-44c8-9a8a-b488554fb809 req-292fa595-9b8c-45d2-ba27-a3368482207a service nova] Creating event network-vif-deleted:f9d86418-8f6a-459d-9f0b-48c8f18b9620 for instance a3f4b771-1726-47a7-9897-bbd002052ab4 on hv-cinder-maste Jul 31 01:26:31.360827 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [req-593539d9-4295-44c8-9a8a-b488554fb809 req-292fa595-9b8c-45d2-ba27-a3368482207a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107296 Jul 31 01:26:31.362155 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 610/1220] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Jul 31 01:26:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 31 01:26:31.685810 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:31.687749 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:31.698858 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:31.699692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:31.700738 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:31.701378 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:31.749930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:31.750725 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:31.751859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.objects.instance [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lazy-loading 'fault' on Instance uuid a3f4b771-1726-47a7-9897-bbd002052ab4 {{(pid=117942) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 31 01:26:31.840131 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.policy [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64fea53260874eae8c13416c97cfa4c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17c952edcc5440989be309944a4c0b00', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=117942) authorize /opt/stack/nova/nova/policy.py:199}} Jul 31 01:26:31.859692 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-26ce8397-32cb-4806-9262-f1d800c71c10 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 200 len: 1327 microversion: 2.1 time: 0.178213 Jul 31 01:26:31.862968 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 611/1221] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:31 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 1327 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 31 01:26:32.888594 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:32.889832 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:32.903557 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:32.904351 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:32.906265 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:32.907097 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:32.933916 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] HTTP exception thrown: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Request method failure captured: Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 HTTP/1.1 Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5djGPdmhK97MxEy13kOqk9WeTKpwfJkusfvj-HpnKAeHO9KeqkJes00Ee5aWZkUHVQtlkPQqLpnFd9iN8cMFpX5bfS-syb_FIQrc1fUXdg9Oj0CAoUi7QcEfg7ojMbPoiMF9LY15L_YS5AVtmLnaJZVk0rwXyYZWKUGHQywFuioXt6g Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:26:32.937919 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/17c952edcc5440989be309944a4c0b00"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_17c952edcc5440989be309944a4c0b00", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 64fea53260874eae8c13416c97cfa4c7 Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:32.941535 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'a3f4b771-1726-47a7-9897-bbd002052ab4'} Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:32.943853 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:32.946700 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:32.949225 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Returning 404 to user: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:32.949225 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-8c7f788c-77a1-4acd-afd6-bb5e0fde7060 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 404 len: 111 microversion: 2.1 time: 0.068754 Jul 31 01:26:32.951454 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 611/1222] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:32 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:26:32.961591 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-2b195090-1041-4bf4-9b62-906f36395e79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '.version_select at 0x7f3e7b4cce50>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:32.986049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-2b195090-1041-4bf4-9b62-906f36395e79 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1917060292" status: 202 len: 0 microversion: 2.1 time: 0.029124 Jul 31 01:26:32.989738 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 612/1223] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Jul 31 01:26:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1917060292 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 31 01:26:36.434944 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:36.436786 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:36.441702 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG dbcounter [-] [117943] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=117943) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:36.454326 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:36.455123 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:36.456587 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:36.457177 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:36.486736 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] HTTP exception thrown: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Request method failure captured: Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 HTTP/1.1 Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5djGPdmhK97MxEy13kOqk9WeTKpwfJkusfvj-HpnKAeHO9KeqkJes00Ee5aWZkUHVQtlkPQqLpnFd9iN8cMFpX5bfS-syb_FIQrc1fUXdg9Oj0CAoUi7QcEfg7ojMbPoiMF9LY15L_YS5AVtmLnaJZVk0rwXyYZWKUGHQywFuioXt6g Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:26:36.489809 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/17c952edcc5440989be309944a4c0b00"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_17c952edcc5440989be309944a4c0b00", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 64fea53260874eae8c13416c97cfa4c7 Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.492459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'a3f4b771-1726-47a7-9897-bbd002052ab4'} Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:36.493377 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.495045 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.496723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Returning 404 to user: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:36.496723 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-aa50d5c3-141d-4a07-a986-1050081e29ea tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 404 len: 111 microversion: 2.1 time: 0.065370 Jul 31 01:26:36.499458 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 612/1224] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:26:36 2022] DELETE /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:26:36.511120 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:36.512323 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: a3f4b771-1726-47a7-9897-bbd002052ab4] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:36.519704 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG dbcounter [-] [117942] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=117942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 01:26:36.535401 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:36.537121 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:36.538670 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:36.539531 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:36.584472 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] HTTP exception thrown: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Request method failure captured: Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 HTTP/1.1 Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5djGPdmhK97MxEy13kOqk9WeTKpwfJkusfvj-HpnKAeHO9KeqkJes00Ee5aWZkUHVQtlkPQqLpnFd9iN8cMFpX5bfS-syb_FIQrc1fUXdg9Oj0CAoUi7QcEfg7ojMbPoiMF9LY15L_YS5AVtmLnaJZVk0rwXyYZWKUGHQywFuioXt6g Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:26:36.587862 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/17c952edcc5440989be309944a4c0b00"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_17c952edcc5440989be309944a4c0b00", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 64fea53260874eae8c13416c97cfa4c7 Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.590930 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'a3f4b771-1726-47a7-9897-bbd002052ab4'} Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:36.592566 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. Jul 31 01:26:36.595371 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:36.597872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Returning 404 to user: Instance a3f4b771-1726-47a7-9897-bbd002052ab4 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:36.597872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-5cf05060-b046-4a22-bdd2-147d0e050b08 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4" status: 404 len: 111 microversion: 2.1 time: 0.089066 Jul 31 01:26:36.597872 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 613/1225] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:36 2022] GET /compute/v2.1/servers/a3f4b771-1726-47a7-9897-bbd002052ab4 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 31 01:26:37.935959 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:37.936907 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.compute.api [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117943) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:37.950263 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:37.951613 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:37.952983 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:37.953779 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG oslo_concurrency.lockutils [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117943) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:37.977210 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.wsgi [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] HTTP exception thrown: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Request method failure captured: Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: request: DELETE /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 HTTP/1.1 Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept: application/json Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Accept-Encoding: identity Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Connection: close Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Length: 0 Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Content-Type: application/json Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Host: 10.222.0.42 Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Openstack-System-Scope: None Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Auth-Token: gAAAAABi5djGPdmhK97MxEy13kOqk9WeTKpwfJkusfvj-HpnKAeHO9KeqkJes00Ee5aWZkUHVQtlkPQqLpnFd9iN8cMFpX5bfS-syb_FIQrc1fUXdg9Oj0CAoUi7QcEfg7ojMbPoiMF9LY15L_YS5AVtmLnaJZVk0rwXyYZWKUGHQywFuioXt6g Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Id: None Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Domain-Name: None Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Identity-Status: Confirmed Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Is-Admin-Project: True Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Id: default Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Domain-Name: Default Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Project-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Role: reader,member Jul 31 01:26:37.978977 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Roles: reader,member Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/17c952edcc5440989be309944a4c0b00"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_17c952edcc5440989be309944a4c0b00", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Id: default Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Domain-Name: Default Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Id: 64fea53260874eae8c13416c97cfa4c7 Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: X-User-Name: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: method: > Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: exception: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:37.983239 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: action_args: {'id': 'ca751a3a-0157-4e91-b8ca-f6594fca8eb9'} Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: {{(pid=117943) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:37.985323 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:37.988650 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: ERROR nova.api.openstack.wsgi Jul 31 01:26:37.991690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: DEBUG nova.api.openstack.wsgi [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Returning 404 to user: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. {{(pid=117943) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:37.991690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO nova.api.openstack.requestlog [None req-cd7456a9-40e9-43c9-9e71-f48d5bb33306 tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 404 len: 111 microversion: 2.1 time: 0.058080 Jul 31 01:26:37.991690 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: [pid: 117943|app: 0|req: 613/1226] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Jul 31 01:26:37 2022] DELETE /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:26:37.997859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Calling method '>' {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 31 01:26:37.997859 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.compute.api [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] [instance: ca751a3a-0157-4e91-b8ca-f6594fca8eb9] Fetching instance by UUID {{(pid=117942) get /opt/stack/nova/nova/compute/api.py:2922}} Jul 31 01:26:38.003045 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:38.003554 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:38.004370 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 31 01:26:38.004822 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG oslo_concurrency.lockutils [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Lock "e9c6d2e4-f70e-4d9b-a2f5-b592dccf0b2d" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=117942) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 31 01:26:38.020867 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.wsgi [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] HTTP exception thrown: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Request method failure captured: Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: request: GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 HTTP/1.1 Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept: application/json Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Accept-Encoding: identity Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Connection: close Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Length: 0 Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Content-Type: application/json Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Host: 10.222.0.42 Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Openstack-System-Scope: None Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: User-Agent: python-urllib3/1.26.10 Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Auth-Token: gAAAAABi5djGPdmhK97MxEy13kOqk9WeTKpwfJkusfvj-HpnKAeHO9KeqkJes00Ee5aWZkUHVQtlkPQqLpnFd9iN8cMFpX5bfS-syb_FIQrc1fUXdg9Oj0CAoUi7QcEfg7ojMbPoiMF9LY15L_YS5AVtmLnaJZVk0rwXyYZWKUGHQywFuioXt6g Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Id: None Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Domain-Name: None Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Identity-Status: Confirmed Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Is-Admin-Project: True Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Id: default Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Domain-Name: Default Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Project-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Role: reader,member Jul 31 01:26:38.022975 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Roles: reader,member Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/17c952edcc5440989be309944a4c0b00"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/17c952edcc5440989be309944a4c0b00"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_17c952edcc5440989be309944a4c0b00", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Id: 17c952edcc5440989be309944a4c0b00 Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-Tenant-Name: tempest-AttachVolumeTestJSON-2026234706 Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Id: default Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Domain-Name: Default Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Id: 64fea53260874eae8c13416c97cfa4c7 Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: X-User-Name: tempest-AttachVolumeTestJSON-2026234706-project Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: method: > Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: exception: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:38.026315 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: action_args: {'id': 'ca751a3a-0157-4e91-b8ca-f6594fca8eb9'} Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: {{(pid=117942) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in get Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2872, in _get_instance Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2829, in _get_instance_from_cell Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 31 01:26:38.029895 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. Jul 31 01:26:38.035532 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: ERROR nova.api.openstack.wsgi Jul 31 01:26:38.040049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: DEBUG nova.api.openstack.wsgi [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] Returning 404 to user: Instance ca751a3a-0157-4e91-b8ca-f6594fca8eb9 could not be found. {{(pid=117942) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 31 01:26:38.040049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO nova.api.openstack.requestlog [None req-58b47205-3015-415a-bf68-dcacd8b971ee tempest-AttachVolumeTestJSON-2026234706 tempest-AttachVolumeTestJSON-2026234706-project] 10.222.0.42 "GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9" status: 404 len: 111 microversion: 2.1 time: 0.034064 Jul 31 01:26:38.040049 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: [pid: 117942|app: 0|req: 614/1227] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Jul 31 01:26:37 2022] GET /compute/v2.1/servers/ca751a3a-0157-4e91-b8ca-f6594fca8eb9 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 31 01:26:38.515221 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:26:38.515676 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:26:38 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:28:46.170106 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:28:46.170794 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Sun Jul 31 01:28:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:28:58.980563 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:28:58.981447 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:28:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:29:58.530069 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:29:58.530806 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:29:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:30:06.096017 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:30:06.096649 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Sun Jul 31 01:30:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:31:39.283933 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:31:39.284898 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:31:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:31:41.619924 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:31:41.620601 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Sun Jul 31 01:31:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:32:06.180523 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:32:06.181459 dv-cinder-master-3-iscsi devstack@n-api.service[117943]: Sun Jul 31 01:32:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 31 01:32:18.991095 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 31 01:32:18.992021 dv-cinder-master-3-iscsi devstack@n-api.service[117942]: Sun Jul 31 01:32:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!